Skipping LifeLock Commercials

Here you can ask your questions on how to use Comskip for the detection of commercials. Also questions on how to remove commercials are welcome
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Skipping LifeLock Commercials

Post by Mike-EEE »

Hello All,

Really digging your product here. I have been diving into it as part of Plex's offering (Plex Lifetime member here).

Everything seems to work well. However, the one stickler I have noticed is that there are these 5-minute infomercials from LifeLock that seem to sneak their way in consistently. I have increased the max_commercial_size from anywhere between 310 and 360 and the max_commercialbreak anywhere from 360 to default of 600 without any luck. I am wondering if there is perhaps another setting I should be fiddling with, so hence the check-in here. Here is my full comskip.ini that I am using, for reference:

Code: Select all

detect_method=107 ; 1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
validate_silence=1 ; Default, set to 0 to force using this clues if selected above.
validate_uniform=1 ; Default, set to 0 to force using this clues (like pure white frames) if blackframe is selected above.
validate_scenechange=1 ; Default, set to 0 to force using this clues if selected above.
verbose=10 ; show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=60 ; frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=40 ; frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=25 ; maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
max_commercialbreak=360 ; maximum length in seconds to consider a segment a commercial break
min_commercialbreak=25 ; minimum length in seconds to consider a segment a commercial break
max_commercial_size=310 ; maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=10 ; mimimum time in seconds for a single commercial
min_show_segment_length=250 ; any segment longer than this will be scored towards show.
non_uniformity=500 ; Set to 0 to disable cutpoints based on uniform frames
max_volume=500 ; any frame with sound volume larger than this will not be regarded as black frame
min_silence=12 ; Any deep silence longer than this amount  of frames is a possible cutpoint
ticker_tape=0 ; Amount of pixels from bottom to ignore in all processing
logo_at_bottom=0 ; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
punish=1 ; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3 ; Multiply when amount is above average * punish_threshold
punish_modifier=4 ; When above average * threshold multiply score by this value
intelligent_brightness=0 ; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA
logo_percentile=0.92 ; if more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.8
punish_no_logo=1 ; Default, set to 0 to avoid show segments without logo to be scored towards commercial
aggressive_logo_rejection=0
connect_blocks_with_logo=1 ; set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable
logo_filter=4 ; set the size of the filter to apply to bad logo detection, 4 seems to be a good value.
cut_on_ar_change=1 ; set to 1 if you want to cut also on aspect ratio changes when logo is present, set to 2 to force cuts on aspect ratio changes. set to 0 to disable
global_threshold=1.79
edge_step=2
delete_show_after_last_commercial=0 ; set to 1 if you want to delete the last block if its a show and after a commercial
delete_show_before_or_after_current=0 ; set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
delete_block_after_commercial=0 ; set to max size of block in seconds to be discarded, set to 0 to disable
remove_before=0 ; amount of seconds of show to be removed before ALL commercials
remove_after=0 ; amount of seconds of show to be removed after ALL commercials
shrink_logo=5 ; Reduce the duration of the logo with this amount of seconds
after_logo=0 ; set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
always_keep_last_seconds=0
padding=1
ms_audio_delay=5
volume_slip=40
skip_b_frames=0 ; Set to 1 to force Comskip to skip frames for higher processing speed.
max_repair_size=200 ; Will repair maximum 200 missing MPEG frames in the timeline, set to 0 to disable repairing for players that don't use PTS.
disable_heuristics=4 ; bit pattern for disabling heuristics, adding 1 disables heristics 1, adding 2 disables heristics 2, adding 4 disables heristics 3, 255  disables all heuristics
delete_logo_file=0 ; set to 1 if you want comskip to tidy up after finishing
output_framearray=0 ; create a big excel file for detailed analysis, set to 0 to disable
output_data=0 ; create a dump of the user data channel, used for CC and XDS (such as V-Chip info). Can be use together with output_framearray to remote debug CC decoding
output_videoredo=0 ; The old videoredo format
output_videoredo3=0 ; The new videoredo v3 format.
output_womble=0
output_mls=0 ; set to 1 if you want MPeg Video Wizard bookmark file output
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_dvrcut=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=1
output_dvrmstb=0 ; Set to 1 if you're running DVRMS-Toolbox
output_edlx=0
output_vcf=0
output_bsplayer=0
output_btv=0 ; set to 1 if you want Beyond TV chapter cutlist output
output_projectx=0 ; set to 1 if you want ProjectX cutlist output (Xcl)
output_avisynth=0
output_vdr=0 ; set to 1 if you want XBMC to skipping commercials
output_demux=0 ; set to 1 if you want comskip to demux the mpeg file while scanning
sage_framenumber_bug=0
sage_minute_bug=0
live_tv=0 ; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4 ; change to 16 when using live_tv in BTV, used for mpeg PS and TS
dvrms_live_tv_retries=300 ; only used for dvr_ms
standoff=0 ; change to 8000000 when using live_tv in BTV
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25  /Z %2 %1"
avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"
dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "
windowtitle="Comskip - %s"
Part of the issue here is that since Plex is a donator license, I haven't been able to debug it as cleanly as I would with a direct donator license. Please see here for more information:
https://forums.plex.tv/t/running-plex-c ... y/226453/5

Thanks again for making this product and for any assistance you can provide!
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

Just put comskip.ini next to the recording and start comskipGUI with one parameter, the recording.
This will allow you to quickly test comskip
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

Cool! From the creator himself. :) Thank you for your quick reply. I did check your manual (http://www.kaashoek.com/files/manual.htm) but do not see anything about ComSkipGui. Is it the same thing as the debug window?

Also to be sure, the Plex version does not contain a ComSkipGui.exe that I can see. I did manage to download it from your source. However, I get the following message when running it:

Code: Select all

h.264 video can only be processed at full speed by the Donator version
h.264 video can only be processed at full speed by the Donator version
It is my understanding that the Plex version is at the Donator level, so pardon my confusion here when attempting to understand all the new pieces presented to me. Is there another setting I should be considering?

Thanks again for your quick assistance and your great work here.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

Yes, same as activating the debug window.

With the donator version you should not get the warning.
But if plex is building its own comskip executable they may have changed something......
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

OK cool... Plex Commercial Skipper.exe does feature the -w switch, so I will see if I can get this working and update here with any findings/results. Thanks again for your valuable time!
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

Alright, after looking at this for a while, it looks like Comskip is simply detecting this as a legitimate block. It registers as block #7, with a length of 300.47s and a Score of .04.

I am thinking that the problem is here is:
  1. This particular episode recording doesn't have a logo, so that's throwing off the detection here.
  2. This LifeLock infomercial gradually fades in, rather than immediately cutting into the commercial like a typical commercial.
I went through about a dozen new recordings using that INI file (which was taken from your USA OTA INI post btw -- viewtopic.php?f=7&t=1741#p8221) and it worked excellently on all of them. I am thinking that this LifeLock thing is that 1% that manages to get by using that configuration. Interestingly enough, it is the last thing that is left on the show, and there isn't any remaining show after it.

FWIW, here's that fade-in action, fading into LifeLock. The show being recorded is Bewitched:
https://youtu.be/laUk5A9pKWo

Unless you can think of a setting that can address that fade and/or account for the non-logo, I am going to say that this is a non-issue and thank you (again) for your work/assistance.

BTW, really nice work on that logo detection. Genius, even. :)
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

YOu could try to set
min_show_segment_length=350
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

erik wrote:YOu could try to set
min_show_segment_length=350
Thank you for the suggestion, @Erik. Unfortunately, it would seem that longer commercials are getting through, not just the LifeLock commercials. It appears that any commercial that is longer than 30 seconds seems to slip in despite the settings. Typically commercials seem to get removed, but when they get fancy like these LifeLock and others, they stay in there. I have tried to set min_show_segment_length to 350 as you suggest and it would seem that these commercials stay in no matter what.

I have reviewed your How to ask for Help link and will be sending you a .zip file of an example. I hope I have provided the correct information. I will also send you a link to my video on cloud storage so you can hopefully view it as well.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

Thanks for the files.
Interesting problem situation.
The text at the bottom of the commercials is decoded as logo.
Therefore you have to disable the block scoring on the presence of logo by setting
logo_present_modifier=1
Logo detection is still value for joining blocks
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

Ok cool! We'll give that a shot and see how it does. Takes a while to get a feel for the success rate so it might be some time before I check-in with status. Thanks again for your help!
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

Alright, seems like this works better, but have found one that was recorded just now with two longer commercials, one of which is LifeLock. xD

As for the other commercial, ComSkip actually detects it on the 2nd time. My thinking here is that Plex is running 0.82.005 whereas the donators version that I just ran this on is 0.82.009. Could it be that somewhere between 005 and 009 that this particular scenario was fixed?

I will be sending you details shortly via email.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

YOu can test hy saving the old plex comskip version and copy the new version over the old one
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

Ah, I did think of that but apparently, 0.82.005 does not exist in the provided donators directory link. :( It would seem that it jumps from 0.82.003 to 0.82.007.

However, I did try 003 and it appears to find the first set of (non-LifeLock) commercials, as well. It did not find the LifeLock commercials, either (to be expected as 009 doesn't find it either with the given ini file).

So, it would seem that an additional run finds commercials that the first run doesn't. Unfortunately, it is unclear on how to accomplish this with Plex's workflow as they seem to perform some extra processing that is unavailable at first glance, shown here:
https://forums.plex.tv/t/running-plex-c ... y/226453/5

All that said, attending to the LifeLock commercial is more important, as I think once we figure that out, I am thinking that other commercials such as the non-LifeLock one we're discussing should also get detected too (hoping :)).
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Skipping LifeLock Commercials

Post by erik »

For commercials such as this long commercial you have to set
min_show_segment_length=350
Mike-EEE
Posts: 12
Joined: Sat Aug 10, 2019 6:38 am

Re: Skipping LifeLock Commercials

Post by Mike-EEE »

AH! A victim of my own tweaking here. I did have that set to 350 at one point and we were still getting longer commercials, so I set it back. That was before the logo_present_modifier, however. So back it goes. Let's see how it does now. 8)
Post Reply