Comskip, EDL files and mencoder

Here you can ask your questions on how to use Comskip for the detection of commercials. Also questions on how to remove commercials are welcome
stattik
Posts: 17
Joined: Sat Dec 24, 2005 9:09 pm

Post by stattik »

The Pentium M laptop came up with the same results. I'm not sure why we're getting different numbers.
stattik
Posts: 17
Joined: Sat Dec 24, 2005 9:09 pm

Post by stattik »

I ran comskip on a true 720P show. The pts-edl file was much more accurate than any other thing I've tried in the past. There is no drifting where the commercial detection gets progressively further off. The beginnings of the commercials breaks are accurate but the endings tend to be of by a commercial of two. Since the preview window doesn't use the pts to display the preview, is there anything else I can do that can be done to help tweak the detection?

One other question. Would it be possible to create a different extension for the pts-edl file (perhaps edlp)? The commercials don't show up in GBPVR when it loads the pts-edl. This way, comskip can create an edl file for gbpvr and a edlp file for mencoder.

Thanks again for all the work.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Post by erik »

Could you set
output_timing=1
in the comskip.ini file and run comskip for a couple of seconds on the .TS file.
This will generate a xxx.timing.csv file.
please post or mail the first 100 lines of that file.
stattik
Posts: 17
Joined: Sat Dec 24, 2005 9:09 pm

Post by stattik »

Code: Select all

type      ,pos       ,framenum  ,framen_inf,exp_frame ,PTS frame ,PTS       ,DTS       ,APTS      ,oframe-PTS,initialPTS,field
settag    ,      1140,         0,         0,         0,         0,   2343493,   2343491,         0,         0,0.000000,   
settag    ,     34564,         0,         0,         0,         0,   2343464,   2343464,         0,         0,0.000000,   
settag    ,     45092,         0,         0,         0,         0,   2343465,   2343465,         0,         0,0.000000,   
settag    ,     54680,         0,         0,         0,         0,   2343468,   2343466,         0,         0,0.000000,   
settag    ,     70848,         0,         0,         0,         0,   2343467,   2343467,         0,         0,0.000000,   
settag    ,     81752,         0,         0,         0,         0,   2343467,   2343467,         0,         0,0.000000,   
settag    ,     91904,         0,         0,         0,         0,   2343471,   2343468,         0,         0,0.000000,   
settag    ,    106380,         0,         0,         0,         0,   2343469,   2343469,         0,         0,0.000000,   
settag    ,    124240,         0,         0,         0,         0,   2343470,   2343470,         0,         0,0.000000,   
settag    ,    135332,         0,         0,         0,         0,   2343473,   2343471,         0,         0,0.000000,   
settag    ,    148868,         0,         0,         0,         0,   2343472,   2343472,         0,         0,0.000000,   
settag    ,    160148,         0,         0,         0,         0,   2343472,   2343472,         0,         0,0.000000,   
settag    ,    170864,         0,         0,         0,         0,   2343476,   2343473,         0,         0,0.000000,   
settag    ,    184212,         0,         0,         0,         0,   2343474,   2343474,         0,         0,0.000000,   
settag    ,    195492,         0,         0,         0,         0,   2343475,   2343475,         0,         0,0.000000,   
settag    ,    199252,         0,         0,         0,         0,   2343488,   2343486,         0,         0,0.000000,   
settag    ,    212412,         0,         0,         0,         0,   2343487,   2343487,         0,         0,0.000000,   
settag    ,    223692,         0,         0,         0,         0,   2343487,   2343487,         0,         0,0.000000,   
settag    ,    235160,         0,         0,         0,         0,   2343491,   2343488,         0,         0,0.000000,   
settag    ,    289868,         0,         0,         0,         0,   2809378,   2809378,         0,         0,0.000000,   
settag    ,    300396,         0,         0,         0,         0,   2809379,   2809379,         0,         0,0.000000,   
change pts,    300396,         0,         0,         0,         0,   2809378,         0,         0,         0,93739.000000,   
v         ,    300396,         0,         0,         0,         0,   2809378,         0,   2809341,         0,93739.000000, B 
settag    ,    319384,         1,         1,         1,4292157919,   2809383,   2809380,         0,4294916224,93739.000000, B 
change pts,    319384,         1,         1,         1,         1,   2809379,         0,         0,         0,93739.000000, B 
v         ,    319384,         1,         1,         1,         1,   2809379,         0,   2809369,         0,93739.000000, B 
settag    ,    332544,         2,         2,         2,         2,   2809381,   2809381,         0,         0,93739.000000, B 
change pts,    332544,         2,         5,         2,         5,   2809383,         0,         0,4294967293,93739.000000, B 
v fill    ,    332544,         2,         5,         2,         5,   2809383,         0,   2809369,4294967293,93739.000000, P 
v fill    ,    332544,         3,         5,         2,         5,   2809383,         0,   2809369,4294967294,93739.000000, P 
v fill    ,    332544,         4,         5,         2,         5,   2809383,         0,   2809369,4294967295,93739.000000, P 
v         ,    332544,         5,         5,         2,         5,   2809383,         0,   2809369,         0,93739.000000, P 
settag    ,    343448,         6,         6,         6,         6,   2809382,   2809382,         0,         0,93739.000000, P 
v         ,    343448,         6,         6,         6,         6,   2809384,         0,   2809369,         0,93739.000000, B 
settag    ,    354728,         7,         7,         7,         7,   2809386,   2809383,         0,         0,93739.000000, B 
v         ,    354728,         7,         7,         7,         7,   2809385,         0,   2809369,         0,93739.000000, B 
settag    ,    367512,         8,         8,         8,         8,   2809384,   2809384,         0,         0,93739.000000, B 
v         ,    367512,         8,         8,         8,         8,   2809386,         0,   2809369,         0,93739.000000, P 
settag    ,    378604,         9,         9,         9,         9,   2809385,   2809385,         0,         0,93739.000000, P 
v         ,    378604,         9,         9,         9,         9,   2809387,         0,   2809369,         0,93739.000000, B 
settag    ,    389884,        10,        10,        10,        10,   2809389,   2809386,         0,         0,93739.000000, B 
v         ,    389884,        10,        10,        10,        10,   2809388,         0,   2809369,         0,93739.000000, B 
settag    ,    402856,        11,        11,        11,        11,   2809387,   2809387,         0,         0,93739.000000, B 
v         ,    402856,        11,        11,        11,        11,   2809389,         0,   2809369,         0,93739.000000, P 
settag    ,    413948,        12,        12,        12,        12,   2809388,   2809388,         0,         0,93739.000000, P 
v         ,    413948,        12,        12,        12,        12,   2809390,         0,   2809369,         0,93739.000000, B 
settag    ,    432560,        13,        13,        13,        13,   2809392,   2809389,         0,         0,93739.000000, B 
v         ,    432560,        13,        13,        13,        13,   2809391,         0,   2809377,         0,93739.000000, B 
settag    ,    445344,        14,        14,        14,        14,   2809390,   2809390,         0,         0,93739.000000, B 
v         ,    445344,        14,        14,        14,        14,   2809392,         0,   2809377,         0,93739.000000, P 
settag    ,    456624,        15,        15,        15,        15,   2809391,   2809391,         0,         0,93739.000000, P 
v         ,    456624,        15,        15,        15,        15,   2809393,         0,   2809377,         0,93739.000000, B 
settag    ,    468092,        16,        16,        16,        16,   2809395,   2809392,         0,         0,93739.000000, B 
v         ,    468092,        16,        16,        16,        16,   2809394,         0,   2809377,         0,93739.000000, B 
settag    ,    524116,        17,        17,        17,        17,   2809393,   2809393,         0,         0,93739.000000, B 
v         ,    524116,        17,        17,        17,        17,   2809395,         0,   2809377,         0,93739.000000, I 
settag    ,    535020,        18,        18,        18,        18,   2809394,   2809394,         0,         0,93739.000000, I 
v         ,    535020,        18,        18,        18,        18,   2809396,         0,   2809377,         0,93739.000000, B 
settag    ,    544608,        19,        19,        19,        19,   2809398,   2809395,         0,         0,93739.000000, B 
v         ,    544608,        19,        19,        19,        19,   2809397,         0,   2809377,         0,93739.000000, B 
settag    ,    568672,        20,        20,        20,        20,   2809396,   2809396,         0,         0,93739.000000, B 
v         ,    568672,        20,        20,        20,        20,   2809398,         0,   2809385,         0,93739.000000, P 
settag    ,    579200,        21,        21,        21,        21,   2809397,   2809397,         0,         0,93739.000000, P 
v         ,    579200,        21,        21,        21,        21,   2809399,         0,   2809385,         0,93739.000000, B 
settag    ,    589352,        22,        22,        22,        22,   2809401,   2809398,         0,         0,93739.000000, B 
v         ,    589352,        22,        22,        22,        22,   2809400,         0,   2809385,         0,93739.000000, B 
settag    ,    603828,        23,        23,        23,        23,   2809399,   2809399,         0,         0,93739.000000, B 
v         ,    603828,        23,        23,        23,        23,   2809401,         0,   2809385,         0,93739.000000, P 
settag    ,    613792,        24,        24,        24,        24,   2809400,   2809400,         0,         0,93739.000000, P 
v         ,    613792,        24,        24,        24,        24,   2809402,         0,   2809385,         0,93739.000000, B 
settag    ,    624884,        25,        25,        25,        25,   2809404,   2809401,         0,         0,93739.000000, B 
v         ,    624884,        25,        25,        25,        25,   2809403,         0,   2809385,         0,93739.000000, B 
settag    ,    644248,        26,        26,        26,        26,   2809402,   2809402,         0,         0,93739.000000, B 
v         ,    644248,        26,        26,        26,        26,   2809404,         0,   2809391,         0,93739.000000, P 
settag    ,    654588,        27,        27,        27,        27,   2809403,   2809403,         0,         0,93739.000000, P 
v         ,    654588,        27,        27,        27,        27,   2809405,         0,   2809391,         0,93739.000000, B 
settag    ,    665680,        28,        28,        28,        28,   2809407,   2809404,         0,         0,93739.000000, B 
v         ,    665680,        28,        28,        28,        28,   2809406,         0,   2809391,         0,93739.000000, B 
settag    ,    679216,        29,        29,        29,        29,   2809405,   2809405,         0,         0,93739.000000, B 
v         ,    679216,        29,        29,        29,        29,   2809407,         0,   2809391,         0,93739.000000, P 
settag    ,    690120,        30,        30,        30,        30,   2809406,   2809406,         0,         0,93739.000000, P 
v         ,    690120,        30,        30,        30,        30,   2809408,         0,   2809391,         0,93739.000000, B 
settag    ,    701212,        31,        31,        31,        31,   2809410,   2809407,         0,         0,93739.000000, B 
v         ,    701212,        31,        31,        31,        31,   2809409,         0,   2809391,         0,93739.000000, B 
settag    ,    756484,        32,        32,        32,        32,   2809408,   2809408,         0,         0,93739.000000, B 
v         ,    756484,        32,        32,        32,        32,   2809410,         0,   2809391,         0,93739.000000, I 
settag    ,    775096,        33,        33,        33,        33,   2809409,   2809409,         0,         0,93739.000000, I 
v         ,    775096,        33,        33,        33,        33,   2809411,         0,   2809398,         0,93739.000000, B 
settag    ,    785812,        34,        34,        34,        34,   2809413,   2809410,         0,         0,93739.000000, B 
v         ,    785812,        34,        34,        34,        34,   2809412,         0,   2809398,         0,93739.000000, B 
settag    ,    799536,        35,        35,        35,        35,   2809411,   2809411,         0,         0,93739.000000, B 
v         ,    799536,        35,        35,        35,        35,   2809413,         0,   2809398,         0,93739.000000, P 
settag    ,    810440,        36,        36,        36,        36,   2809412,   2809412,         0,         0,93739.000000, P 
v         ,    810440,        36,        36,        36,        36,   2809414,         0,   2809398,         0,93739.000000, B 
settag    ,    821532,        37,        37,        37,        37,   2809416,   2809413,         0,         0,93739.000000, B 
v         ,    821532,        37,        37,        37,        37,   2809415,         0,   2809398,         0,93739.000000, B 
settag    ,    835068,        38,        38,        38,        38,   2809414,   2809414,         0,         0,93739.000000, B 
v         ,    835068,        38,        38,        38,        38,   2809416,         0,   2809398,         0,93739.000000, P 
settag    ,    846160,        39,        39,        39,        39,   2809415,   2809415,         0,         0,93739.000000, P 
v         ,    846160,        39,        39,        39,        39,   2809417,         0,   2809398,         0,93739.000000, B 
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Post by erik »

Stattik,

I get the save .timing.csv as you have,
Now I need to get smarter to see where the problem is between the PTS reconstruction and the EDL calculation.
The suggestion for .edlp seems a good one, otherwise it confuses everybody.
uryan
Posts: 3
Joined: Sat May 05, 2007 12:34 am

Post by uryan »

Hi Eric,
I have had a similar problem with a edlp producing bad values:

Code: Select all

-1927.88	-1818.52	0
-1447.32	-1252.08	0
-697.84	-517.76	0
-229.84	-84.76	0
9.12	79.44	0
I generated a timing.csv file and it had something interesting towards the end:

Code: Select all

type      ,pos       ,framenum  ,framen_inf,exp_frame ,PTS frame ,PTS       ,DTS       ,APTS      ,oframe-PTS,initialPTS,field
...
settag    ,1663273940,     48190,     48190,     48190,     48190,   2386086,   2386083,         0,         0,93515.000000, B 
v         ,1663273940,     48190,     48190,     48190,     48190,   2386085,         0,   2386084,         0,93515.000000, B 
v         ,1663333532,     48191,     48191,     48191,     48191,   2386086,         0,   2386086,         0,93515.000000, I 
v         ,1663361168,     48192,     48192,     48192,     48192,   2386087,         0,   2386086,         0,93515.000000, B 
v         ,1663388804,     48193,     48193,     48193,     48193,   2386088,         0,   2386087,         0,93515.000000, B 
v         ,1663428284,     48194,     48194,     48194,     48194,   2386089,         0,   2386087,         0,93515.000000, P 
v         ,1663455920,     48195,     48195,     48195,     48195,   2386090,         0,   2386089,         0,93515.000000, B 
v         ,1663487504,     48196,     48196,     48196,     48196,   2386091,         0,   2386089,         0,93515.000000, B 
v         ,1663523036,     48197,     48197,     48197,     48197,   2386092,         0,   2386090,         0,93515.000000, P 
v         ,1663550672,     48198,     48198,     48198,     48198,   2386093,         0,   2386091,         0,93515.000000, B 
v         ,1663578308,     48199,     48199,     48199,     48199,   2386094,         0,   2386092,         0,93515.000000, B 
v         ,1663605944,     48200,     48200,     48200,     48200,   2386095,         0,   2386092,         0,93515.000000, P 
v         ,1663625684,     48201,     48201,     48201,     48201,   2386096,         0,   2386093,         0,93515.000000, B 
settag    ,1663646556,     48202,     48202,     48202,     48202,         6,         3,         0,         0,93515.000000, B 
v         ,1663646556,     48202,     48202,     48202,     48202,   2386097,         0,   2386093,         0,93515.000000, B 
change pts,1663688852,     48203,     48203,     48203,     48203,         6,         0,         0,         0,-1927.000000, B 
v         ,1663688852,     48203,     48203,     48203,     48203,         6,         0,   2386095,         0,-1927.000000, I 
v         ,1663700696,     48204,     48204,     48204,     48204,         7,         0,   2386095,         0,-1927.000000, B 
v         ,1663724384,     48205,     48205,     48205,     48205,         8,         0,   2386096,         0,-1927.000000, B 
v         ,1663752020,     48206,     48206,     48206,     48206,         9,         0,   2386098,         0,-1927.000000, P 
v         ,1663775708,     48207,     48207,     48207,     48207,        10,         0,   2386098,         0,-1927.000000, B 
v         ,1663799396,     48208,     48208,     48208,     48208,        11,         0,         0,         0,-1927.000000, B 
v         ,1663842824,     48209,     48209,     48209,     48209,        12,         0,         2,         0,-1927.000000, P 
v         ,1663854668,     48210,     48210,     48210,     48210,        13,         0,         2,         0,-1927.000000, B 
v         ,1663886252,     48211,     48211,     48211,     48211,        14,         0,         3,         0,-1927.000000, B 
v         ,1663917836,     48212,     48212,     48212,     48212,        15,         0,         6,         0,-1927.000000, P 
v         ,1663941524,     48213,     48213,     48213,     48213,        16,         0,         7,         0,-1927.000000, B 
settag    ,1663966344,     48214,     48214,     48214,     48214,        18,        15,         0,         0,-1927.000000, B 
v         ,1663966344,     48214,     48214,     48214,     48214,        17,         0,         7,         0,-1927.000000, B 
v         ,1664012588,     48215,     48215,     48215,     48215,        18,         0,         9,         0,-1927.000000, I 
v         ,1664024432,     48216,     48216,     48216,     48216,        19,         0,        10,         0,-1927.000000, B 
v         ,1664048120,     48217,     48217,     48217,     48217,        20,         0,        10,         0,-1927.000000, B 
If i adjust the entries in the edlp file by 95442 (-1927 - 93515) then it seems to be at the right points

ProjectX does not show any problems at the point where the problem occurred. If you want I can upload a small section of the file containing this point. (where would be the best place to upload to?)
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Post by erik »

No need to upload
Its clear that the PTS during the recording is reset. That is what also happens with certain Hauppauge PVR150's (like the one I have)
When the edlp is intended to position on the PTS and the PTS is not continues than I am afraid it will not work.
This because comskip remembers the last calculated start PTS and I do not know how to use multiple PTS ranges as they could overlap.....
uryan
Posts: 3
Joined: Sat May 05, 2007 12:34 am

Post by uryan »

Ah, yes it's obvious now that you mention it :P

I did some testing with MEncoder, and it always writes the actual PTS when creating an EDL, but it will not properly read back the EDL it created. It drops any entries that are not sequencial.

In short, I don't think theres anything you can do. It's MEncoders problem.
stattik
Posts: 17
Joined: Sat Dec 24, 2005 9:09 pm

Re: Comskip, EDL files and mencoder

Post by stattik »

Hi Erik,

I'm revisiting this thread because I've ran into a number of TS recordings in which the PTS resets in the middle of the file. Is there anything in the comskip logfile that would help me identify when this happens? I've scanned it and didn't see any correlation. If not, would it be possible to add it? If I know a file has it's PTS reset, I could remux it and use the EDL file for cutting.
erik
Site Admin
Posts: 3368
Joined: Sun Aug 21, 2005 3:49 pm

Re: Comskip, EDL files and mencoder

Post by erik »

Stattik,

When you set
output_timing=1
a large .timing.csv is create and that will show ant jumps in the base pts. (that is initial pts) as this initial pts is recalculated for every frame.

Another possibility is to set
verbose=10
and scan the log file for
"pts jumped"
when the jump is too big to repair then a warning is given that the timeline is corrupt and that editing will not be reliable
stattik
Posts: 17
Joined: Sat Dec 24, 2005 9:09 pm

Re: Comskip, EDL files and mencoder

Post by stattik »

Thanks Erik. After looking though the various files I decided to analyze the EDLP file. If the values are negative or greater than 95440, I remux and use the EDL file to cut.
Post Reply