Did I request INI tuning help correctly?
Posted: Wed Feb 06, 2008 2:57 pm
Hi Erik, hi everyone,
9 days ago I followed the procedure listed at "How to ask for help" at viewtopic.php?f=5&t=34 and posted two ZIP attachments and a covering email to erik@kaashoek.com.
Perhaps I should have posted my query here first, as a courtesy?
(If so, I apologise, and will paste the contents of the email below, just to be sure. I'll also re-send the email right now, in case it got lost somehow.)
It's just that after 9 days I would have thought some kind of response might be expected, and my MPGs which need Comskipping are piling up!
I'm NOT complaining about this brief delay, though. Far from it! I'd simply thought I would save all the other forum readers the tedium of reading my long, boring, covering email by dealing directly with you (as per the "how to ask for help" instructions.)
But who knows, perhaps some readers may find it helpful too.
I look forward to a reply at your earliest convenience, Erik, and reading any suggestions you can make regarding my INI settings. Keep up the great work!
Columbus
Fremantle, Western Australia
Here's the paste of that January 28 email:
Hi Erik,
let me add my voice to the chorus: "well done on a great piece of software, I hope it makes you rich one day, you deserve it!"
I too would appreciate a bit of guidance regarding fine-tuning my INI file ... it's working pretty well now, but I'm a bit baffled as to how to fine-tune it better from here, and I have one additional, unusal problem I'm hoping you can help with.
I've endeavoured to follow your instructions at "How to ask for help" at viewtopic.php?t=34... please note I've attached TWO zips, containing two job samples using the identical INI file. The INI is fairly closely based on your "More aggressive Comskip INI" at viewtopic.php?f=2&t=427&p=1884&hilit=ag ... +ini#p1884 , but not identical.
• One show is a 4:3 black-bordered "Futurama" episode. Comskip's accuracy with this one is pretty good.
• The other is a 16:9 widescreen "Mythbusters" episode, Comskip is having trouble with one of the ad breaks on this one.
Beginning / ending accuracy of both isn't exactly right either, although the "Futurama" one is mostly darn close.
I've included the comskip.ini, the .csv, .log and .ref files as instructed, and I've carefully edited the REF files to indicate my desired frame cut locations. I've also included the logo.txt and original (unedited) .txt files as Comskip generated them.
The reason for including both the TXT and REF files?
Well, I've included both the TXT and edited REF files so you can directly compare them and see another, particular dilemma I have: a constant problem Comskip has with ALL detections on ANY recordings of mine is this: there is always a spurious "ad break" detected where none exists, at the VERY END of each show; the very last few seconds in fact.
(In other words, both TXT file examples show an entire, EXTRANEOUS FINAL LINE, for an ad-break that never exists, with a duration varying from about 5 to about 140 frames only, compared to the REF files, which are corrected and what I would LIKE Comskip to generate, with your help.)
I have no idea what causes this, and no experimenting I've done with the INI variables seems to enforce my simple desire: "do not detect a spurious ad break under 5 seconds duration AT THE VERY END OF THE RECORDING, IT IS NOT AN AD BREAK."
Can you think of anyway of "hard-wiring" my INI to enforce this? I can manually edit the lines out, but that defeats my near-flawless automation of this whole process. (Note shows here, such as "Futurama" often have a final, single 60 seconds of program content, immediately before this bogus final, ad detection.)
I wonder if any other users suffer from this problem? Would you like me to post about it in your forum to find out?
I'm a contributor (in a small way) to Paul Ahb / Paul Bartlett's very handy WSMPP project ( 'Web Scheduler Module Post Processor' ) ... which is basically a clever collection of VBS scripts and batch files interfacing with several open source video processing modules (e.g. ProjectX) which are highly customisable and can be made to do ANYTHING to your video files with ANY command-line addressable software.
Have you heard of it? Comskip has become a valued component of it, and I was one of its earliest supporters, and one of the first to have a go at INI tuning.
Basically, these are the steps I take:
• record SD DVB-TV in .TS (full Transport Stream) mode using Web Scheduler
• trim the head and tail of the resultant TS file with Mpg2Cut2 ( a fault tolerant, GOP level, binary MPG2 cutter )
then, using WSMPP:
• demux to MPG2 with ProjectX (thus fixing timeline glitches / errors and extracting perfect .SUB subtitles)
• remux back to MPG2 with mplex1
• run Comskip on the new MPG2 (command line) generating EDL (for MPlayer) and BCF (for BSPlayer)
• convert to very high quality MP3 / DivX using DGIndex, WaveGain, lame, AviSynth, avs2avi, DivX Pro 6.8 and finally DivXMux
This all works beautifully, although it's taken a long time to perfect.
The flexibility of AviSynth scripting, of course, basically means any conceivable cropping / processing / resizing / title overlaying etc can be done, and the resultant AVI DivX file can make use of your Comskip ad break files (i.e. EDLs with MPlayer, or my favourite: BCFs with BSPlayer.) The end result is very slick indeed ... when Comskip gets its break detection correct!
(And yes, I could even use Comskip to create an AviSynth file for hard-cutting of the output DivX file, but I can't do that 'cos I wish to use the previously-extracted subtitles with their hard-wired timeline, without having to permanently burn them into the final AVI, also: the ad-break detection isn't 100% reliable as of yet.) Now a utility that could re-synch the subtitle's timeline to match a comskip edit list on the fly ... hmm, that would be something!
I only mention all that in case it's of interest to you, or might explain how the errant final, bogus 5 second "ad breaks" are being detected ... is it to do with all this timeline manipulation / correction? I can't see how, but I mention all these boring details just in case.
I'm hoping, ultimately, that the one INI file can cover all my channels' quirks ... I've selected the two attached samples as very representative of the free-to-air DVB-TV we have here in Perth, Western Australia (although they're very different program types, aspect ratios and from different channels). The logos' positions, sizes and durations are typical of usage here.
So, in short, I'm making two requests:
• Please, can you tweak my INI settings, or point out where my current settings are failing?
• Please, is there any way of nuking that errant last few seconds ad-break entry?
Along with everyone else, I appreciate the hard work you've put into this wonderful tool, which is so close to perfect, it's tantalising!
Erik, If I can help you with anything at anytime, don't hesitate to ask. I await your response with bated breath.
Cheers from Fremantle, Western Australia.
Columbus
(70mm cinema projectionist, film / tv post-production technician for many years, general-purpose I.T. guy, MCP, CISCO CCNA Academy graduate, corporate network engineer)
P.S. URLs of software referred to above:
Web Scheduler ( see http://forums.dvbowners.com/index.php?showforum=28/ and http://www.bluebit.com.au/ )
Mpg2Cut2 ( see http://www.geocities.com/rocketjet4/ )
WSMPP project ( 'Web Scheduler Module Post Processor' ) at http://forums.dvbowners.com/index.php?s ... topic=6888
Attachments were:
--------------------------------------------------------------------------------
4ACV01 Futurama COMSKIP test.zip 582.0 kb
--------------------------------------------------------------------------------
(01-21) SBS - Mythbusters COMSKIP test.zip 1163.8 kb
9 days ago I followed the procedure listed at "How to ask for help" at viewtopic.php?f=5&t=34 and posted two ZIP attachments and a covering email to erik@kaashoek.com.
Perhaps I should have posted my query here first, as a courtesy?
(If so, I apologise, and will paste the contents of the email below, just to be sure. I'll also re-send the email right now, in case it got lost somehow.)
It's just that after 9 days I would have thought some kind of response might be expected, and my MPGs which need Comskipping are piling up!
I'm NOT complaining about this brief delay, though. Far from it! I'd simply thought I would save all the other forum readers the tedium of reading my long, boring, covering email by dealing directly with you (as per the "how to ask for help" instructions.)
But who knows, perhaps some readers may find it helpful too.
I look forward to a reply at your earliest convenience, Erik, and reading any suggestions you can make regarding my INI settings. Keep up the great work!
Columbus
Fremantle, Western Australia
Here's the paste of that January 28 email:
Hi Erik,
let me add my voice to the chorus: "well done on a great piece of software, I hope it makes you rich one day, you deserve it!"
I too would appreciate a bit of guidance regarding fine-tuning my INI file ... it's working pretty well now, but I'm a bit baffled as to how to fine-tune it better from here, and I have one additional, unusal problem I'm hoping you can help with.
I've endeavoured to follow your instructions at "How to ask for help" at viewtopic.php?t=34... please note I've attached TWO zips, containing two job samples using the identical INI file. The INI is fairly closely based on your "More aggressive Comskip INI" at viewtopic.php?f=2&t=427&p=1884&hilit=ag ... +ini#p1884 , but not identical.
• One show is a 4:3 black-bordered "Futurama" episode. Comskip's accuracy with this one is pretty good.
• The other is a 16:9 widescreen "Mythbusters" episode, Comskip is having trouble with one of the ad breaks on this one.
Beginning / ending accuracy of both isn't exactly right either, although the "Futurama" one is mostly darn close.
I've included the comskip.ini, the .csv, .log and .ref files as instructed, and I've carefully edited the REF files to indicate my desired frame cut locations. I've also included the logo.txt and original (unedited) .txt files as Comskip generated them.
The reason for including both the TXT and REF files?
Well, I've included both the TXT and edited REF files so you can directly compare them and see another, particular dilemma I have: a constant problem Comskip has with ALL detections on ANY recordings of mine is this: there is always a spurious "ad break" detected where none exists, at the VERY END of each show; the very last few seconds in fact.
(In other words, both TXT file examples show an entire, EXTRANEOUS FINAL LINE, for an ad-break that never exists, with a duration varying from about 5 to about 140 frames only, compared to the REF files, which are corrected and what I would LIKE Comskip to generate, with your help.)
I have no idea what causes this, and no experimenting I've done with the INI variables seems to enforce my simple desire: "do not detect a spurious ad break under 5 seconds duration AT THE VERY END OF THE RECORDING, IT IS NOT AN AD BREAK."
Can you think of anyway of "hard-wiring" my INI to enforce this? I can manually edit the lines out, but that defeats my near-flawless automation of this whole process. (Note shows here, such as "Futurama" often have a final, single 60 seconds of program content, immediately before this bogus final, ad detection.)
I wonder if any other users suffer from this problem? Would you like me to post about it in your forum to find out?
I'm a contributor (in a small way) to Paul Ahb / Paul Bartlett's very handy WSMPP project ( 'Web Scheduler Module Post Processor' ) ... which is basically a clever collection of VBS scripts and batch files interfacing with several open source video processing modules (e.g. ProjectX) which are highly customisable and can be made to do ANYTHING to your video files with ANY command-line addressable software.
Have you heard of it? Comskip has become a valued component of it, and I was one of its earliest supporters, and one of the first to have a go at INI tuning.
Basically, these are the steps I take:
• record SD DVB-TV in .TS (full Transport Stream) mode using Web Scheduler
• trim the head and tail of the resultant TS file with Mpg2Cut2 ( a fault tolerant, GOP level, binary MPG2 cutter )
then, using WSMPP:
• demux to MPG2 with ProjectX (thus fixing timeline glitches / errors and extracting perfect .SUB subtitles)
• remux back to MPG2 with mplex1
• run Comskip on the new MPG2 (command line) generating EDL (for MPlayer) and BCF (for BSPlayer)
• convert to very high quality MP3 / DivX using DGIndex, WaveGain, lame, AviSynth, avs2avi, DivX Pro 6.8 and finally DivXMux
This all works beautifully, although it's taken a long time to perfect.
The flexibility of AviSynth scripting, of course, basically means any conceivable cropping / processing / resizing / title overlaying etc can be done, and the resultant AVI DivX file can make use of your Comskip ad break files (i.e. EDLs with MPlayer, or my favourite: BCFs with BSPlayer.) The end result is very slick indeed ... when Comskip gets its break detection correct!
(And yes, I could even use Comskip to create an AviSynth file for hard-cutting of the output DivX file, but I can't do that 'cos I wish to use the previously-extracted subtitles with their hard-wired timeline, without having to permanently burn them into the final AVI, also: the ad-break detection isn't 100% reliable as of yet.) Now a utility that could re-synch the subtitle's timeline to match a comskip edit list on the fly ... hmm, that would be something!
I only mention all that in case it's of interest to you, or might explain how the errant final, bogus 5 second "ad breaks" are being detected ... is it to do with all this timeline manipulation / correction? I can't see how, but I mention all these boring details just in case.
I'm hoping, ultimately, that the one INI file can cover all my channels' quirks ... I've selected the two attached samples as very representative of the free-to-air DVB-TV we have here in Perth, Western Australia (although they're very different program types, aspect ratios and from different channels). The logos' positions, sizes and durations are typical of usage here.
So, in short, I'm making two requests:
• Please, can you tweak my INI settings, or point out where my current settings are failing?
• Please, is there any way of nuking that errant last few seconds ad-break entry?
Along with everyone else, I appreciate the hard work you've put into this wonderful tool, which is so close to perfect, it's tantalising!
Erik, If I can help you with anything at anytime, don't hesitate to ask. I await your response with bated breath.
Cheers from Fremantle, Western Australia.
Columbus
(70mm cinema projectionist, film / tv post-production technician for many years, general-purpose I.T. guy, MCP, CISCO CCNA Academy graduate, corporate network engineer)
P.S. URLs of software referred to above:
Web Scheduler ( see http://forums.dvbowners.com/index.php?showforum=28/ and http://www.bluebit.com.au/ )
Mpg2Cut2 ( see http://www.geocities.com/rocketjet4/ )
WSMPP project ( 'Web Scheduler Module Post Processor' ) at http://forums.dvbowners.com/index.php?s ... topic=6888
Attachments were:
--------------------------------------------------------------------------------
4ACV01 Futurama COMSKIP test.zip 582.0 kb
--------------------------------------------------------------------------------
(01-21) SBS - Mythbusters COMSKIP test.zip 1163.8 kb