The OBE blog
Warning: like Part One of this series, these posts are very technical!
After converting the old MMX simple IDCT of FFmpeg from inline assembly to external (as described in Part One) I was to look at making the IDCT faster. A naive approach is to convert from directly using the mm registers to using xmm registers. This can usually be done with minimal changes just paying attention to packs, unpacks, and moves. This can make things faster on Skylake and related microarches from Intel. A discussion of why is beyond the scope of this post. The point is that you can measure that functions are faster if they use xmm registers.
A lot of what we do with our C-100 and C-200 decoders involves decoding MPEG-2 video which is still prevalent in spite of being over 25 years old. We're often also using professional profiles like 4:2:2 which hardware decoders on CPUs and GPUs can't cope with. Looking at the decode process in perf shows a slow IDCT written in MMX (yes, really!), so we sent one of engineers, James to make it faster and modernise the code. Here’s how he did it in his own words (be warned, it is about to get very technical!):
Week’s 7 and 8 have been merged (again) owing to various trips abroad. One highlight was being able to visit the Mobile TV Group UHD/HDR truck. This truck was doing Basketball for FOX Sports and we learnt how they are working on testing HDR for various broadcasters. We also show how UHD streams were managed and the challenges with cable overload and managing 2SI (sample interleave) vs quadrants.
Nice to be able to visit this pic.twitter.com/MVpxcQ1PWW— OpenBroadcastSystems (@OpenBroadcastSy) February 16, 2017
In Week 8 we started testing SMPTE 2022-6 with our colleagues @skynewstech:
We’re learning a lot about how to deploy software 2022-6 streams ourselves in a multivendor environment. More on this at our NAB BEITC speech "Don’t Just Go IP, Go IT".
At Open Broadcast Systems, we have a lot of repetitive business processes that essentially involve the same simple tasks every day or every week. We like to follow the famous Japanese continuous improvement process of Kaizen, made famous by Toyota, where every employee from the factory floor worker to the CEO can suggest and enact improvements to their work to improve efficiency. But in the era of automation, we want to do more - we want to improve processes by several orders of magnitude and then have them done automatically. This lets us run with a much lower administrative headcount compared to similar companies, as well as move more quickly.
One of the hardest processes to improve was our ordering processes for Blackmagic cards. Owing to recent GBP currency fluctuations, our supply chain has had regular price changes, most of which we automatically calculate from USD or EUR and pull into spreadsheets.
However, all Blackmagic equipment from the UK distributor is priced using a master PDF price list pictured below:
It is, of course, easy for humans to read this datasheet but for a machine, it has a confusing mix of pictures, description, and a large number of subcategories of device. Note how there could be a single heading to list the subcategories or just a single device type. None of this is easy for a computer to understand.
Initially we tried basic PDF extraction tools like pdfextract but they struggled with the complex table structure. But then we found Tabula, software used by journalists to parse released documents. It was able to understand the document structure very well as we can see:
From there it was some very simple python scripting to extract prices and loading all this data into our supply chain spreadsheets. We can now run this as a batch job every day and have nicely updated prices.
We want to do this for as many business processes, from the simple to the complicated. However, we lack a lot of things to really make the most of automation: