Page 1 of 2

ATSC 3.0 Workaround (with 4k?)

Posted: Mon Sep 27, 2021 2:47 am
by TMC
I see there is another thread discussing the issue of 4k videos. (I have actual ATSC 3.0 recordings)
I found that Comskip has a problem with ATSC 3.0 - 1080p/720p (HEVC). As there are no stations broadcasting any higher than 1080/720p in my area I can’t verify if my workaround works with 4k (I think it should because of the scaling), perhaps someone else can confirm.
The problem: Comskip appears to process these recordings fine but it always states “Commercials were not found.” and he edl file is 0bytes.
I‘ve found the following command lines to work successfully,
Windows:
ffmpeg -hide_banner -v error -i BBhevc.ts -vf scale=480:-1 -c:v mpeg4 -f matroska pipe: | comskip -q --ini=C:\Bin\comskip\comskip.ini --output-filename=BBhevc pipe: .
Linux:
ffmpeg -hide_banner -v error -i BBhevc.ts -vf scale=480:-1 -c:v mpeg4 -f matroska pipe: | comskip -q --ini=/usb/m/comskip.ini --output-filename=BBhevc pipe: .
I tried other ffmpeg setting but they either result the wrong output in the .edl or random broken pipes. The biggest downside to this is the time penalty ~4x in windows and whopping ~16x on linux (PI 4b, it takes about the same amount of time for Comskip to run as it does to watch it).

Any hope this can be done better natively by Comskip?
I can supply a sample video link if Erik is interested.
Thanks,
TMC

Video info
Title 720p HEVC
Codec HEVC
Profile Main 10
Level 120
Resolution 1280x720
Aspect Ratio 16:9
Interlaced No
Framerate 59.94
Bitrate 1,922 kbps
Bit Depth 10 bit
Pixel Format yuv420p10le
Reference Frames 1

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sat Oct 02, 2021 3:19 pm
by tzr916
Ah, some progress. Really wish Erik would help us with this! Again, I would donate $100 USD to get it working.

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sat Oct 02, 2021 7:55 pm
by AlphaZulu
I have some 4K 60FPS FIOS material, (Captured by an Ceton), I believe it's MPEG2

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Tue Feb 15, 2022 8:59 am
by erik
I release comskip 82.011 using the 4.3 release of ffmpeg and doubled maximum video size
Can someone test and if this fails send me a test video?

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Wed Feb 16, 2022 7:37 pm
by AlphaZulu
I'm running a really fast and dirty test now. (Expanded files into folder and dump a 4K recording on comskipGUI).

It's running very very slow, but that may be from the generic settings, BUT unlike build .10, it has not crashed yet.

I'll report back later, if/when it finishes.

Thanks Eric.

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Thu Feb 17, 2022 9:31 pm
by AlphaZulu
I had to stop the run due to time, but I'll get to proper testing next week.

I can say while it ran, it never crashed. I think I stopped it at 45 min...

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Fri Feb 25, 2022 1:40 am
by tmm1
TMC wrote: Mon Sep 27, 2021 2:47 am Bit Depth 10 bit
Pixel Format yuv420p10le
Reference Frames 1
I added support for 10-bit video in https://github.com/erikkaashoek/Comskip/pull/143

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Fri Feb 25, 2022 3:13 am
by TMC
Good!

I can confirm that this is version does work with the 2 tests I’ve tried (720p 10bit & 1080i 10bit). edit HEVC(265)
Test were done on a R_Pi4 (Linux mypi 5.15.23-v8+ #1525 SMP PREEMPT Wed Feb 16 14:36:10 GMT 2022 aarch64 GNU/Linux)

[sudo apt-get -y install autoconf automake libtool git build-essential libargtable2-dev libavformat-dev libsdl1.2-dev libsdl2-dev
cd
rm -rf Comskip
git clone git://github.com/erikkaashoek/Comskip \
&& cd Comskip \
&& ./autogen.sh \
&& ./configure \
&& make -j$(nproc)\
&& sudo make install]

The speed may be a little (or more) better than my ‘Workaround’ but I won’t give any #’s as my server is doing other stuff now.

Everybody TEST! (4K?)
Time to donate?? (again)…

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sat Feb 26, 2022 3:52 am
by TMC
Further testing the slowness of this seams to be that Comskip is pushing almost everything through 1 thread(core) no matter how many are enabled
(thread_count=<int> in INI file or --threads=<int> on command line)
This results in only about 15% of my (windows 10) CPU(s) being used.
Can this be improved/fixed?
Also --hwassist and/or --cuvid does not make my NVIDIA GTX 1660 help as I would think HEVC decoding would be improved by this.
Any way good work!

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sat Feb 26, 2022 6:30 pm
by AlphaZulu
On the speed of processing; there is definitely something going on here (build .12). and @TMC might be on to something with single thread use. It's very slow for me.

12 cores / Intel(R) Core(TM) i7-8700K CPU

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sat Feb 26, 2022 6:31 pm
by erik
I'm afraid the HW accelerator does not work anymore. Could not find yet why it is not activated

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sun Feb 27, 2022 2:11 am
by TMC
I've now noticed that 'comskip82_012.zip' in http://www.kaashoek.com/files/ is the "Public build" could this be why comskip is running single threaded on windows?

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Thu Mar 03, 2022 10:37 pm
by AlphaZulu
As far as I can tell, both builds .11 and .12 have the issue.

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Tue Mar 08, 2022 3:16 am
by tmm1
The 10-bit -> 8-bit conversion code is single threaded.

I think swscale has some newer APIs for threading, so it's possible it can be made multi-threaded. It will require some investigation and I don't have time at the moment.

EDIT: Looks like ffmpeg 5.0 would be required to use it: https://github.com/FFmpeg/FFmpeg/commit/d6fdc78e9164c8e6bf7bfc7766932c467b874aa2

Currently the project does not compile with ffmpeg 5.0, so that's the first step for anyone who wants to help make this happen.

Re: ATSC 3.0 Workaround (with 4k?)

Posted: Sun Aug 14, 2022 1:10 am
by TMC
Seams to be unofficially working better (ffmpeg 5+ and libswscale5)
Much better performance on ubuntu 22.04 and even better with --cuvid
Although i get random crashes (I say random because retrying usually works)

Here's my compile script:

Code: Select all

#!/bin/bash                                                                                                                                                       
                                                                                                                                                                  
sudo apt update -q                                                                                                                                                
sudo apt -y install \                                                                                                                                             
  autoconf \                                                                                                                                                      
  automake \                                                                                                                                                      
  libtool \                                                                                                                                                       
  git \                                                                                                                                                           
  build-essential \                                                                                                                                               
  libargtable2-dev \                                                                                                                                              
  libavformat-dev \                                                                                                                                               
  libsdl1.2-dev \                                                                                                                                                 
  libsdl2-dev \                                                                                                                                                   
  libswscale-dev \                                                                                                                                                
  libswscale5                                                                                                                                                     
                                                                                                                                                                  
cd                                                                                                                                                                
rm -rf Comskip                                                                                                                                                    
git clone https://github.com/erikkaashoek/Comskip \                                                                                                               
  && cd Comskip \                                                                                                                                                 
  && ./autogen.sh \                                                                                                                                               
  && ./configure \                                                                                                                                                
  && make -j$(nproc)\                                                                                                                                             
  && sudo make install