Looping on Boardwalk Empire

[jstach500]

Hello everyone,

I love Comskip but lately it appears to be getting hung up on certain HBO programs. Here is an example log file for Boardwalk Empire. It never gets around to creating the XML file. Any thoughts on what I can do to fix this? I have 3 workers running, but eventually they all get into loops on HBO programs so none of my other shows end up being processed. At least I think that's what is happening based on the log files and looking at the creation dates and times for the XML files that Comskip does create.

Code: Select all

Start Process: 9/21/2014 : 8:59 PM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 9840
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=8:59 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=8:59 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:00 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:00 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:01 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:01 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:02 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:02 PM LN=522977280
Waiting to process file: FT = 8:59 PM WT = 9:04 PM CT=9:03 PM LN=522977280
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:03 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:04 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:04 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:05 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:05 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:06 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:06 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:07 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:07 PM LN=971243520
Waiting to process file: FT = 9:03 PM WT = 9:08 PM CT=9:08 PM LN=971243520
Queue file
Duration = 00:09:31.3271453
Waiting for available worker: 9/21/2014 9:08 PM
Worker allocated: 9/21/2014 9:08 PM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.4327077
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 9/22/2014 : 4:00 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 3420
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:01.0313036
Waiting for available worker: 9/22/2014 4:00 AM
Worker allocated: 9/22/2014 4:00 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.4647450
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 9/23/2014 : 4:07 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 3096
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:01.0116743
Waiting for available worker: 9/23/2014 4:07 AM
Worker allocated: 9/23/2014 4:11 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.5589278
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 9/24/2014 : 4:00 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 3464
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:03.3256266
Waiting for available worker: 9/24/2014 4:00 AM
Worker allocated: 9/24/2014 4:07 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.4953301
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 9/25/2014 : 4:00 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 1504
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:01.0278747
Waiting for available worker: 9/25/2014 4:00 AM
Worker allocated: 9/25/2014 4:06 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.4753177
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 9/26/2014 : 4:00 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 3504
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:01.0098223
Waiting for available worker: 9/26/2014 4:00 AM
Worker allocated: 9/26/2014 4:06 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.4913284
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 
Start Process: 10/3/2014 : 11:12 AM
InputFile=D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
Processing actions, this may take some time!!
Current Affinity: 15 System Mask: 15
Getting MetaData
Testing "Processed Files"
Testing "Every Sage File"
Testing "Every File"
"Every File" is a match.
Run Profile = nativecommdetect
ProcessId = 2612
DvrmsToolbox Version: 1.2.2.6
Throttle DVRMStoMPEG
FWThrottle: D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv
WaitTime: 5 PollTime
Waiting to process file
Queue file
Duration = 00:00:01.0106751
Waiting for available worker: 10/3/2014 11:12 AM
Worker allocated: 10/3/2014 11:17 AM
Get Video Information
704x480 : 16x9 ATSC=True
Duration = 00:00:00.5293531
Find Commercials in file
Create Analyzer: Comskip
Use Comskip
Using comskip
Running comskip this may take a while.
Run C:\Program Files (x86)\DVRMSToolbox\Applications\comskip.exe "D:\Recorded TV\Boardwalk Empire_HBOHD_2014_09_21_20_59_00.wtv" "C:\Users\Public\DvrmsToolbox\CommercialsXml" 

Thanks for your help.

[erik]

Can you post a comskip generated log file?

Try to run comskip manually
Drag and drop a recording on comskip.exe
A command window should appear and you should see comskip is processing.

[jstach500]

Erik,

Thanks for your message. I tried running Comskip manually. In the process I noticed a few interesting things. One is that my Boardwalk Empire log files were huge. The largest was over 177 megabytes. I deleted the old log file for one of the episodes and manually ran Comskip. As it ran, it repeatedly said the stream was encrypted and that decoding may fail. I can view the show in WMC, so it appears to decode properly on my system. I am not sure if that statement in the Comskip console window was truly a problem or just a warning. The big issue appeared later, where the console window running Comskip now just repeats the phrase "Panic volume buffer." The log file continues to grow, but there does not appear to be anything of substance happening. I may try reverting to an earlier version to Comskip to see if that fixes the issue. I'll let you know how that goes, but in the meantime, if you think of anything that would help, please let me know.

Thank you.

[erik]

Comskip can not and will never be able to decode encrypted files.
WMC can play encrypted files as microsoft has made a deal with the content owners it will never enable copying the content without encryption, only playback to a HDMI or VGA output.
Although comskip does not output the content "in the clear" I for sure wil never be able to make the same deal with the content owners.

[jstach500]

Thanks Erik. That makes sense. The puzzling part to me is that Comskip was never hung up on HBO files before, or at least I had not noticed it being an issue. The way it is hanging up now is preventing it from scanning many of my other files. Once I reach my maximum number of workers with Comskip tied up on HBO files, it can never move on. Any tips on how to prevent this? Can I set a long time limit where Comskip will exit and write a dummy .xml file so DVRMSToolbox won't keep calling Comskip to process those files?

Thanks again for your help. I'll let you know how my tests go with an earlier Comskip version.

[jstach500]

I reverted to donator version 058 and it looks like I have the same issue. It has been running all night and the log files are getting large. The strange thing is that on September 19, 2014, Comskip output an XML file for one of my Boardwalk Empire recordings. Now it is looping on recordings of the same show from the same channel that were recorded both before and after the one that succeeded. Maybe this information can help diagnose the issue. Thanks for considering all of this for me.

[erik]

Can you make one of those protected files available to me so I can see if I can change comskip to skip such a file?

[erik]

Thanks for the file.

I added functionality in build 0.81.066 to detect a protected file after some time and abort instead of an endless loop.
Can you test?

[jstach500]

Outstanding! Yes, I would be glad to test. I'll try to do it tonight and can let you know in the morning how it worked. Thanks so much!

[jstach500]

Erik,

Wonderful job! DVRMSToolbox is now able to move on to other files for Comskip to scan. The one question I have is about the file output when Comskip aborts on these files. It looks like it outputs a corresponding txt and vih file in the commercials folder. Do you know if the presence of these files will prevent DVRMSToolbox from initiating another instance of Comskip to scan the files, or does it need to see an xml file to prevent this from happening?

Thanks so much for all your help.

[erik]

It depends on the rules you use for dvrmstb

[joesc1]

This is great news! Thanks Erik, it's something we've been requesting a while. But to save processing cycles, can you look at the file metadata to look for the "WM/WMRVContentProtected" flag to determine if it should run on protected files? If it finds the file is protected, just create a blank xml and log file and skip the scan by comskip?

[erik]

That's for the future as certain files start non protected and then become protected later during the recording.

[joesc1]

Ah, good point. Definitely a problem for live tv. Are you using the Panic volume buffer error to do this detection when a show is live? Or just periodically checking the file metadata to see if the protected flag changes?

[erik]

I check if the ratio between audio frames and video frames is "reasonable".
This also catches some other decoding problems.