Auto-detect commercials via stills

anthonyshephard · Post by **anthonyshephard** » Thu Feb 22, 2007 9:34 pm

Hello,
It didn't seem to work all that well, but perhaps I don't know what I'm doing

I changed the ini to 128 for cut scene only, then dragged and dropped onto comskipGUI.exe. I was surprised it was processing as it had nothing to do. After 10 minutes I could go through and choose a cut frame. I did this, saved a frame, and ran the processing again. Unfortunately no commercials were detected. Looking through the log file, and looking at the blocks in the GUI it is fairly successful at detecting blocks, but not 100%. One of the blocks detected is only 2.28 seconds in length. The first frame is wrong, it is earlier in the sequence than all the others (the relocation relocation is sliding across, relocati on top, location on bottom arrow). After the 2.28 seconds it correctly matches. I'm not quite sure why it would do this, as it is successful with all the other matches.

So, how to I get these blocks correctly weighed for commercial?
I tried to add all relevant detection options (method = 237), but although some commercials are now detected, the block detection is less accurate. I don't think this can be the correct method.

Any suggestions would be appreciated.

A small observation, you appear to have forgotten linefeed after the F1 help for letter c (it runs into F2 key description). Plus the log file description of detection method is missing the new cut scene method

Anthony

Post by **erik** » Fri Feb 23, 2007 7:52 am

YOu could also have dragged and dropped a previously generated .csv or .txt , as long as the preview was working.

Yes, you can, and most probably should, combine methods, at least 43+128=171

Could you set verbose=10 and post the generated log file?
Or even better, could you set output_framearray=1 and mail me the generated .csv file together with your ini file?
COuld you include in the mail where the commercials are supposed to be?

I currently do not use the cutframes for scoring, only for cutframe detection.
When I receive your .csv file i will look into the possibility to use the cutframes also for scoring.

anthonyshephard · Post by **anthonyshephard** » Fri Feb 23, 2007 1:30 pm

Hi,
Yes of course I'll send you the details. However it won't be until Tuesday as I am going away for the weekend.
I should say, do you want the log file & csv for the suggested detection method, just the cut scene method, or both?

Thanks

Anthony

Post by **erik** » Fri Feb 23, 2007 1:55 pm

For the suggested method, 171

anthonyshephard · Post by **anthonyshephard** » Tue Feb 27, 2007 8:47 pm

Hi, Logs have been sent to your email as requested.

Anthony

Post by **erik** » Wed Feb 28, 2007 2:47 pm

Thanks,

I did some testing, most seems OK, the error is caused by a to high threshold for the cutscene detection. Try to lower or use another cutscene.

Next to that I tried if I could tune the default comskip to work on your recording.

This comskip.ini based on the default version seems to do a good job without cutscenes. I inserted blank lines around the changes. Could you verify?

Code: Select all

detect_method=111			;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
validate_silence=1			; Default, set to 0 to force using this clues if selected above.
validate_uniform=1			; Default, set to 0 to force using this clues (like pure white frames) if blackframe is selected above.
validate_scenechange=1		; Default, set to 0 to force using this clues if selected above.
verbose=10				;show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=60      			;frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=40      		;frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=25			;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
max_commercialbreak=600 		;maximum length in seconds to consider a segment a commercial break

min_commercialbreak=60			;minimum length in seconds to consider a segment a commercial break

max_commercial_size=125			;maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=4   		;mimimum time in seconds for a single commercial
min_show_segment_length=125 	; any segment longer than this will be scored towards show.
non_uniformity=500
max_volume=500				; any frame with sound volume larger than this will not be regarded as black frame
min_silence=20				; Any deep silence longer than this amount  of frames is a possible cutpoint
ticker_tape=0				; Amount of pixels from bottom to ignore in all processing 
logo_at_bottom=0			; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
punish=0					; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3		; Multiply when amount is above average * punish_threshold
punish_modifier=2			; When above average * threshold multiply score by this value
intelligent_brightness=0 		; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA
logo_percentile=0.92			; if more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.75
punish_no_logo=1			; Default, set to 0 to avoid show segments without logo to be scored towards commercial
aggressive_logo_rejection=0
connect_blocks_with_logo=1		; set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable

cut_on_ar_change=2			; set to 1 if you want to cut also on aspect ratio changes, set to 0 to disable

delete_show_after_last_commercial=1	; set to 1 if you want to delete the last block if its a show and after a commercial

delete_show_before_first_commercial=1

delete_show_before_or_after_current=0	; set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
delete_block_after_commercial=0	;set to max size of block in seconds to be discarded, set to 0 to disable 
remove_before=0				; amount of seconds of show to be removed before ALL commercials
remove_after=0				; amount of seconds of show to be removed after ALL commercials
shrink_logo=5				; Reduce the duration of the logo with this amount of seconds
after_logo=0		; set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
padding=0
ms_audio_delay=5
max_repair_size=200			; Will repair maximum 200 missing MPEG frames in the timeline, set to 0 to disable repairing for players that don't use PTS. 
delete_logo_file=0			; set to 1 if you want comskip to tidy up after finishing
output_framearray=0			; create a big excel file for detailed analysis, set to 0 to disable
output_videoredo=0
output_womble=0
output_mls=0			; set to 1 if you want MPeg Video Wizard bookmark file output
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_dvrcut=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=0
output_edlx=0
output_vcf=0
output_bsplayer=0
output_btv=0				; set to 1 if you want Beyond TV chapter cutlist output
output_projectx=0			; set to 1 if you want ProjectX cutlist output (Xcl)
output_avisynth=0
output_demux=0				; set to 1 if you want comskip to demux the mpeg file while scanning
sage_framenumber_bug=0
sage_minute_bug=0
live_tv=0					; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=1			; change to 16 when using live_tv in BTV
standoff=0					; change to 8000000 when using live_tv in BTV
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25  /Z %2 %1"
avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"
dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "
windowtitle="Comskip - %s"

anthonyshephard · Post by **anthonyshephard** » Wed Feb 28, 2007 10:01 pm

Hi Eric,
Your recommended ini file works quite well.

Start of show, very good (i.e. removing blocks 0 and 1). Frame is actually 2599, estimate is 2595
Start of first commercial is wrong (i.e. removing block 5). There is a fade to black that is not transfer into commercial. From 13796 to 14547 is show
Show starts again at frame 19998, so correctly starting at block 14.
Advert again at 44352, so correctly removing block 16 – 21, as show starts at 50548.
Advert correctly identified at block 23 – 31.
Show starts at 32, and end of show correctly identified and removed 34-37.

So only one problem and this is block 5 being scored as commercial when it is show.

This appears to be caused by fuzzy logic;
Block 5 has strict standard length for a commercial.
Block 5 score: Before - 1.00 After - 9.00

I have emailed you the logs;

PS I should also say, I am confused by your comment that the detection threshold is too high. Will this not cause fewer detections rather than more?
Also, you may wish the change the ini file as according to the comments cut_on_ar_change=2 is not a valid option

Anthony

Post by **erik** » Thu Mar 01, 2007 7:11 am

anthonyshephard wrote:I am confused by your comment that the detection threshold is too high. Will this not cause fewer detections rather than more?

Indeed, but that is what you need when using only stills because the error is caused by a wrong identification. Lowering the threshold, try
cutscene_threshold=10

Also, you may wish the change the ini file as according to the comments cut_on_ar_change=2 is not a valid option

Indeed, its an undocumented option, maybe I should document it.

anthonyshephard · Post by **anthonyshephard** » Sat Mar 03, 2007 9:57 pm

Hello,
I tried again with a different show, updating the cut scenes. But I did not have much success. I have emailed you the log files in case you are interested.
Anthony

Post by **erik** » Sat Mar 03, 2007 10:21 pm

With the newst build you can set
non_uniformity=0
which removes the large error of show being classified as commercial.

It is strange to see not one cut scene match?????
Did you review the result with the comskipGUI to check the matching results?

anthonyshephard · Post by **anthonyshephard** » Sun Mar 04, 2007 10:05 pm

Hi,
Yes, for some strange reason, no matching scenes (just running 128 method). The is the first time I have tried with more than 1 (three in this case). This is going up to 40 cut scene threshold. As I had success with 10 I would have thought this would be okay. Any suggestions?

Also, you have mentioned another undocumented option

. I tried it but I did not see much improvement.

Anthony

Post by **erik** » Tue Mar 06, 2007 8:05 am

No suggestion. I tried also with upto 3 cutscenes in parallel and that worked for me.
The cutscene line in the debugwindow went to zero for all three recognized frames.

Maybe you are experiencing a lot of noise or artifacts in your recording?

anthonyshephard · Post by **anthonyshephard** » Tue Mar 06, 2007 6:19 pm

No, the recording is fine, plus I tried it on this weeks gadget show and the same problem. Is there any limitation for duration of cutscene? The frames are only present for a short while.

Is it mentioned somewhere which graph is for cutscenes?

Anthony

Post by **erik** » Tue Mar 06, 2007 6:58 pm

From the readme.txt

Changes in 0.79.73
-- New cutpoint detection method: CutScenes, method number 128
First record one or more frames to be used as cutscene by running ComskipGUI on the MPEG, move to the frame you want to use as cutscene and hit 'C'. Select where you want the frame to be saved.
Then use up to 8 cutsceneframes in your ini file
cutscenefile1="C:\\Work\\cutscene1.dmp"
cutscenefile2="C:\\Work\\cutscene2.dmp"
Set the cutscene threshold higher if you have low quality video.
cutscene_threshold=20
Using more cutscenes will slowdown Comskip, when cutscenes are enabled the Debugwindow will show in the pink line cutscene information instead of scene change information

Or in other words:
A cutscene is actually a cutframe so one frame in length
Comskipgui shows the match with the cutframes in the pink line. The lower the more match. On a perfect match (cutframe from the same recording) the line should go all the way down. Setting the threshold to 20 means the any line that goes beyond 20% from the bottom is a match.

anthonyshephard · Post by **anthonyshephard** » Tue Mar 06, 2007 9:46 pm

Thanks for the information. .
I think it is working then, but perhaps not matching the right scenes. The pink line does drop at certain times, and turns white which I think means matching cutscenes. But they don't look that good. Unfortunately while reviewing I managed to get a crash in DGIndex, AppName: comskipgui.exe, v 1.4.6.0, Offset 0001cb650. I have copied the DMP file in case you find this useful. I will email it to you.

My latest theory is that the cut scenes are in general correct but the GUI review is not. Do you have enough room on your server for the file, 1,346 Mb?

Anthony