Re-processing Live TV

Here you can ask your questions on how to use Comskip for the detection of commercials. Also questions on how to remove commercials are welcome
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Re-processing Live TV

Post by kiwijunglist »

I have comskip currently set to process a file while it is being recorded.

I am using livetv=1 in the comskip.ini file.

The readme file mentions that
;This will enable the output of commercial information during the processing allowing the skipping of detecting commercials before the processing is finished
;;Use when you want to process during recording and have the commercial information available before the recording is finished. This uses a simplified detection algorithm so the results may be different from the final results.
Does using livetv=1 have any impact on the final results, ie. would I get a better final result if I wasn't running comskip live and I wasn't using livetv=1?
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Re: Re-processing Live TV

Post by kiwijunglist »

I did a test where I processed a file during a recording
then reprocessed the same file using the exact same settings when the recording finished
the process during recording detected 1x add at 1:30
reprocessing with the same settings detected 3x adds

I have a feeling that comskip is finishing the processing before the stream has finished recording despite me using

live_tv=1
live_tv_retries=16
dvrms_live_tv_retries=1200
standoff=40960
dvrmsstandoff=120000


another question - is it possible to specifiy transport stream switch (-t) in the comskip.ini
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Processing Live TV

Post by kiwijunglist »

ok I seem to be unable to get comskip to work on live recordings. It looks like it stops after only a couple of minutes of processing. The logs show no errors, it's as though it thinks the recording has finished.

I am using "comskip -t -d 79 recording.ts" on a live recording

I have the following settings in comskip.ini

live_tv=1
live_tv_retries=16
dvrms_live_tv_retries=1200
standoff=40960
dvrmsstandoff=240000

http://www.sendspace.com/filegroup/52tf ... p7tQHVaH2P
^ download my comskip.ini, ts_recording.log, ts_recording.txt

PS. If I run comskip after a recording is finished it works 100% perfectly.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Re-processing Live TV

Post by erik »

What application do you use to record?

Some applications, such as DBViewer, make it impossible for comskip to process during recording
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Re: Re-processing Live TV

Post by kiwijunglist »

Sorry I should of written that I am using Mediaportal TV Server 1.2

EDIT: and i'm recording DVB-T H264 Progressive Video, saved as mpeg transport stream (I think it's 480P, but I'd have to double check)

EDIT2: Also with the help of someone else we will be updating the comskip launcher plugin for tvserver to add additional options (custom comskip.ini generated by MP) as well a as updating the MP wiki

EDIT3: I should also add that I noticed the default comskip.ini has detect_method=41 that overrides any command line switch, initially I had trouble figuring out why comskip wasn't changing the detection method with the command line switch.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Re-processing Live TV

Post by erik »

I'm testing a new comskip build that at least for NPVR is confirmed to work well with processing a TS during recording.
Hope to release coming days.
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Re: Re-processing Live TV

Post by kiwijunglist »

thanks, I guess I incorrectly assumed that it was working with live ts recordings via media portal. I'm happy to provide testing.
Jagad
Posts: 131
Joined: Wed Dec 07, 2011 12:37 am

Re: Re-processing Live TV

Post by Jagad »

@kiwijunglist

How does MediaPortal handle "Last Modification Time" during a recording (if you study the explorer fileproperties while recording)? NPVR updates this timestamp continuously (every second or so) during the whole recording. DVBViewer doesn't update this timestamp at all during a recording, but the update occurs when the recording finishes at first.
It's intresting to know, cuz I believe the update frequency of this timestamp is crucial for Comskip livetv processing of ts files as it is designed today.
kiwijunglist
Posts: 19
Joined: Mon Oct 22, 2012 10:23 am

Re: Re-processing Live TV

Post by kiwijunglist »

MP TV-SERVER RECORDING: Data Modified and Date Created (as seen in windows explorer) are set to the time and date of the start of the recording. Data modified remains static during the recording, but does get updated at the end of the recording.

If comskip wanted to it could just keep trying to process the file until date modified =/= date created. Then you could get rid of all the retry settings etc.
Jagad
Posts: 131
Joined: Wed Dec 07, 2011 12:37 am

Re: Re-processing Live TV

Post by Jagad »

Sorry for the late response. Thanks for doing the modification time check :) It seems we are in the same boat and this is the normal way of windows file handling afaik. The usual file handling is open-write-close and modification time gets updated after close. Seems NPVR deviates from the standard. That should imply Comskip can't detect if a recording is ongoing if using Media Portal and DVBViewer as PVRs. Every recording status check returns as finished. It's here the recording engine behaviour comes into play. The inner workings of the DVBViewer recording engine is as follows:
"The whole file writing is threaded. If the disk load is too high and the data isn't written fast enough it might be buffered internally and written later."
That should mean as soon as Comskip starts to TS liveprocess, the file writing of DVBViewer stops (cuz load increases). This causes Comskip to think the recording is finished and close too early, because modification time check returns static. Comskip is probably coded to interpret a static modification time as recording is finished. It may be Media Portal uses a similar recording behaviour.

So, using timestamps as status indicator if recording is ongoing or finished simply isn't possible, because we have these file handling differences between our PVR softwares. Instead a common denominator has to be choosen, which regardless of PVR software behaves the same. The only one that springs to my mind would be filesize. Filesize never stays static if the recording is ongoing and if Comskip constantly monitor the filesize within a sensible timedelta, that would be the optimal recording-status-indicator, which would if still growing force Comskip to, regardless of threaded writing and write pause during high diskload, keep up and finish the liveprocessing properly.

We'll see what Erik comes up with. After all the Media Portal and DVBViewer PVRs has a huge amounts of users that would be very happy if Comskip liveprocessing of TS files could work alright.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Re-processing Live TV

Post by erik »

File size is ok as long as I know at what interval the file is written.
How long to wait?
5 seconds?
1 minute?
5 minutes?
Jagad
Posts: 131
Joined: Wed Dec 07, 2011 12:37 am

Re: Re-processing Live TV

Post by Jagad »

Hi Erik!

Yes, that is a very good question...setting a sensible timedelta that works for every PVR system is tricky.
My proposal, after studying the timings of DVBViewer write pause during high diskload, would be to create an "interval of filesize check" variable that could be tweakable from comskip.ini. With a default value of approximately 5 seconds should be enough, based on the write pause never lasted any longer than that during highest possible diskload. If this isn't enough there's always the chance to increase the filesize check interval or decrease it if using some other PVR software that uses smaller recording buffers.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Re-processing Live TV

Post by erik »

Will do
dvasco
Posts: 6
Joined: Wed Dec 13, 2006 6:26 pm

Re: Re-processing Live TV

Post by dvasco »

erik wrote:I'm testing a new comskip build that at least for NPVR is confirmed to work well with processing a TS during recording.
Hope to release coming days.
I am really looking forward to this.
Jagad
Posts: 131
Joined: Wed Dec 07, 2011 12:37 am

Re: Re-processing Live TV

Post by Jagad »

Hi Erik!

Played around with the new build 81_049 and more specific TS LiveTV processing on my DVBViewer system yesterday and to my surprise it worked with default settings (without the standoff settings). Comskip Live processing does no longer abort prematurely and exits gracefully 15 seconds after recording has finished. I was surprised it worked cuz as far as I know my proposal has not been implemented yet, yes? If not how did you get Comskip to detect if recording is ongoing or finished on DVBViewer and Mediaportal PVR's then?

Just intrested to know and nonetheless many thanks for a build very well executed :D

Regards
Jagad
Post Reply