Request - Working ini for New Zealand Freeview

To discuss the tuning of comskip and for posting your ini files
papapumpnz
Posts: 3
Joined: Wed Apr 29, 2009 8:25 am

Request - Working ini for New Zealand Freeview

Post by papapumpnz »

Hi,

Anyone have a working ini that detects the commerical breaks for New Zealands Freeview? Im having a hard time getting it to reliably detect commericals on the H.264 streams.
DrSoftware
Posts: 1
Joined: Thu Jul 02, 2009 4:28 am

Re: Request - Working ini for New Zealand Freeview

Post by DrSoftware »

I'm about to embark on this process, so I'm very interested to know how far you've got and am willing to put my two cents worth in try and make it work. I'm finding that it seems to find the logos fine, but the sections it's skipping are offset by almost the entire commercial break. I'm also getting alot of errors during processing of the ts file - as follows, I'd be interested to know if you're getting these as well.

Unsopported AAC format, sf_index > 11
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: PCE shall be the first element in a frame
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Channel coupling not yet implemented
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Channel coupling not yet implemented
faac: frame decoding failed: Unexpected fill element with SBR data
faac: frame decoding failed: Channel coupling not yet implemented
faac: frame decoding failed: Unexpected fill element with SBR data
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Channel coupling not yet implemented
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Maximum number of bitstream elements exceeded
faac: frame decoding failed: Bitstream value not allowed by specification
faac: frame decoding failed: Channel coupling not yet implemented
faac: frame decoding failed: Unexpected channel configuration change
faac: frame decoding failed: Invalid number of channels
faac: frame decoding failed: Channel coupling not yet implemented
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

Certain channels in NZ use a form of AAC encoding that comskip can not decode.
This will degrade the performance of Comskip
wittsend
Posts: 2
Joined: Tue Jul 06, 2010 8:48 am

Re: Request - Working ini for New Zealand Freeview

Post by wittsend »

i've been working on one for quite a while now and it works properly sometimes.. mainly on Prime.
As for audio that can't be helped for now, but as long as you get it finding the blackframes and logo properly, and have the add and show segment lengths right it tends to work half the time. The main problem I'm having at the moment is the comskip seems to ignore performing the scoring on some blocks and they come out with a score of 1.00... just under the threshold so some adds still come through. Anyways heres my INI file so far for NZ TV.. I use with .TS files from mediaportal. Give it a go and see what you think.. remember its still a farcry from perfect so don't give me shit please.. maybe you can help me improve it.

**edit mucked around with the heuristics tonight to make it rely more on the presence of a logo as well as edited the show segment and max commercial lengths.. seems to be better, but still having problems detecting the logo on TV1. it seems to be quite faint compared to the other channels

Code: Select all

detect_method=43			;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all
validate_silence=1			; Default, set to 0 to force using this clues if selected above.
validate_uniform=1			; Default, set to 0 to force using this clues (like pure white frames) if blackframe is selected above.
validate_scenechange=1		; Default, set to 0 to force using this clues if selected above.
verbose=10				;show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=61      			;frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=41     		;frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=21			;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
max_commercialbreak=310 		;maximum length in seconds to consider a segment a commercial break
min_commercialbreak=25			;minimum length in seconds to consider a segment a commercial break
max_commercial_size=150			;maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=3   		;mimimum time in seconds for a single commercial
min_show_segment_length=311 	; any segment longer than this will be scored towards show.
non_uniformity=40			; Set to 0 to disable cutpoints based on uniform frames
max_volume=250				; any frame with sound volume larger than this will not be regarded as black frame
min_silence=12				; Any deep silence longer than this amount  of frames is a possible cutpoint
ticker_tape=0				; Amount of pixels from bottom to ignore in all processing 
logo_at_bottom=0			; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting
punish=0					; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3		; Multiply when amount is above average * punish_threshold
punish_modifier=2			; When above average * threshold multiply score by this value
intelligent_brightness=0 		; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA
logo_percentile=0.92			; if more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.75
punish_no_logo=0			; Default, set to 0 to avoid show segments without logo to be scored towards commercial
aggressive_logo_rejection=0
connect_blocks_with_logo=1		; set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable
logo_filter=4               ; set the size of the filter to apply to bad logo detection, 4 seems to be a good value.
cut_on_ar_change=0			; set to 1 if you want to cut also on aspect ratio changes when logo is present, set to 2 to force cuts on aspect ratio changes. set to 0 to disable
delete_show_after_last_commercial=0	; set to 1 if you want to delete the last block if its a show and after a commercial
delete_show_before_or_after_current=0	; set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
delete_block_after_commercial=0	;set to max size of block in seconds to be discarded, set to 0 to disable 
remove_before=0				; amount of seconds of show to be removed before ALL commercials
remove_after=0				; amount of seconds of show to be removed after ALL commercials
shrink_logo=5				; Reduce the duration of the logo with this amount of seconds
after_logo=0		; set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
padding=0
ms_audio_delay=5
volume_slip=40
skip_b_frames=0				; Set to 1 to force Comskip to skip frames for higher processing speed.
max_repair_size=200			; Will repair maximum 200 missing MPEG frames in the timeline, set to 0 to disable repairing for players that don't use PTS. 
disable_heuristics=4		bit pattern for disabling heuristics, adding 1 disables heristics 1, adding 2 disables heristics 2, adding 4 disables heristics 3, 255  disables all heuristics 
delete_logo_file=0			; set to 1 if you want comskip to tidy up after finishing
output_framearray=1			; create a big excel file for detailed analysis, set to 0 to disable
output_debugwindow=1
output_data=0				; create a dump of the user data channel, used for CC and XDS (such as V-Chip info). Can be use together with output_framearray to remote debug CC decoding
output_videoredo=0
output_womble=0
output_mls=0			; set to 1 if you want MPeg Video Wizard bookmark file output
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_dvrcut=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=0
output_edlx=0
output_vcf=0
output_bsplayer=0
output_btv=0				; set to 1 if you want Beyond TV chapter cutlist output
output_projectx=0			; set to 1 if you want ProjectX cutlist output (Xcl)
output_avisynth=0
output_vdr=0				; set to 1 if you want XBMC to skipping commercials
output_demux=0				; set to 1 if you want comskip to demux the mpeg file while scanning
sage_framenumber_bug=0
sage_minute_bug=0
live_tv=0					; set to 1 if you use parallelprocessing and need the output while recording
live_tv_retries=4			; change to 16 when using live_tv in BTV, used for mpeg PS and TS
dvrms_live_tv_retries=300			; only used for dvr_ms
standoff=0					; change to 8000000 when using live_tv in BTV
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25  /Z %2 %1"
avisynth_options="LoadPlugin(\"MPEG2Dec3.dll\") \nMPEG2Source(\"%s\")\n"
dvrcut_options="dvrcut \"%s.dvr-ms\" \"%s_clean.dvr-ms\" "
windowtitle="Comskip - %s"
edge_step=2
edge_radius=2
give_up_logo_search=30000
edge_level_threshold=2
length_strict_modifier=3.0
length_nonstrict_modifier=1.5
combined_length_strict_modifier=2.0
combined_length_nonstrict_modifier=1.25
logo_present_modifier=0.01
excessive_length_modifier=0.01
ar_wrong_modifier=2.0
min_schange_modifier=0.5
max_schange_modifier=2.0
schange_threshold=80
cc_commercial_type_modifier=4.0
cc_wrong_type_modifier=2.0
cc_correct_type_modifier=0.75
dictionary_modifier=1.05
global_threshold=0.99
Last edited by wittsend on Thu Sep 02, 2010 7:56 am, edited 1 time in total.
wittsend
Posts: 2
Joined: Tue Jul 06, 2010 8:48 am

Re: Request - Working ini for New Zealand Freeview

Post by wittsend »

Update: Found one recording that missed one block of ads because the logo started before the blackframe on C4 (meaning the blocks were joined either side of the blackframe making the length too long to write off as ads). I upped the shrink_logo to 10

Sound works with TV3 as this channel uses AC3 rather than AAC

Comskip works well with TV3, C4, Prime, but not so well with TV1 yet.. Not sure about other channels yet. When I get more recordings done on the other channels I'll know.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

Once you get stuck follow the "how to ask for help" post and I will have a look.
sheppy99
Posts: 88
Joined: Mon May 10, 2010 2:44 am

Re: Request - Working ini for New Zealand Freeview

Post by sheppy99 »

I'm also trying to get something that works in NZ. Please let me know if you make any progress!
Cheers
Kevin
sheppy99
Posts: 88
Joined: Mon May 10, 2010 2:44 am

Re: Request - Working ini for New Zealand Freeview

Post by sheppy99 »

I've discovered that Videoredo4 can recode the audio from AAC to MP2 so that Comskip can read it.
Currently playing with the ini file kindly provided by "wittsend" to make it more reliable on my system.
So far I use DVRMSToolbox to convert everything from WTV (I use Media Center) to TS for processing, for all channels apart from TV3 which has DD sound I convert the TS again to TS with H264 and MP2 sound. Once this is done comskip reads both the sound and vision.
My current problem is getting reliable logo detection, it finds the logo every time but it struggles to accurately detect when it has disappeared.
Has anyone found a solution to our faint logos?
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

Experiment with
logo_filter
Values ranging from 0 (default) to max 10
sheppy99
Posts: 88
Joined: Mon May 10, 2010 2:44 am

Re: Request - Working ini for New Zealand Freeview

Post by sheppy99 »

Thanks. I've increased it to 6 so far and it looks better.
I'm struggling to use the debug window. Is there a way of stepping through the video once it's scanned so I can see the individual frames and what is detected with the actual video in the background?
I can see the logo line roughly following the commercial / program but it isn't a straight drop during the commercial and sometimes comes back during the middle of the break, I'd like to look at this on a frame by frame basis if possible so I can tune further. I've moved to a different ini per channel as the commercial lengths and shrink_logo settings work better on a channel by channel basis.
Does skip_b_frames=2 work on HDTV 720P and 1080i? I'm trying to make the tuning a little faster.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

Are the instructions in debugwindow.htm inside the comskip distribution zip file not working?
sheppy99
Posts: 88
Joined: Mon May 10, 2010 2:44 am

Re: Request - Working ini for New Zealand Freeview

Post by sheppy99 »

Not sure I understood them - will reread tomorrow and report back.
The window appears when I do the initial scan and remains after but maybe I'm using the wrong keys but I can't step back and forward with the video behind. Then when I use the excel file to do the scan I have no video to step through, just a black screen with multiple graphs
Does this sound right?
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

When the excell file is in the same folder as the original video the video should be shown behind the graphs while moving through the video (clicking just above the video in the window)
sheppy99
Posts: 88
Joined: Mon May 10, 2010 2:44 am

Re: Request - Working ini for New Zealand Freeview

Post by sheppy99 »

Just tried it again - no video. Command line used:
U:\Comskip>comskip --debugwindow "u:\temp_tv2\Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.csv"
The top part of the logfile has this in it:
################################################################
Generated using Comskip 0.80.031
Loading comskip csv file - u:\temp_tv2\Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.csv
Time at start of run:
Fri Aug 13 13:18:11 2010
################################################################
Mpeg: u:\temp_tv2\Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.ts
Exe comskip.exe
Logo: u:\temp_tv2\Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.logo.txt
Ini: comskip.ini

In the same folder is Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.ts
and executing comskip with:
U:\Comskip>comskip --debugwindow "u:\temp_tv2\Ramsay's Kitchen Nightmares USA_TV ONE_2010_08_12_21_30_00.ts"
causes it to run with the decoded video in the background whilst it's running.

Any ideas what I should do next to get the debug window to show the video whilst using the CSV file?
I've tried it on another machine and it's the same, the video also disappears from the debug window on the original scan once it's finished.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Request - Working ini for New Zealand Freeview

Post by erik »

Could you try comskipGUI.exe instead of comskip.exe?
No need for the --debugwindow switch then
Not sure it will make a big difference
Post Reply