[AI 소개서] ① 2 weeks with ‘AI upscaling’ picture quality improvement, AI only 3 hours

Companies are pouring out services using artificial intelligence (AI) technology. As such, AI has already penetrated deeply into our lives. We looked at what AI services are easily accessible in everyday life and how to use them. [편집자주]

Although 4K displays have been commercialized, many of the contents provided by YouTube or OTT (Over The Top) sometimes do not have the same quality. As in this case, there is a technology that is attracting attention because it can improve a low-pixel image to fit a larger display. That’s upscaling.

Upscaling improves the quality of an image or video by filling in missing pixels. This technology can be divided into AI and non-AI upscaling. Non-AI upscaling, which was used before AI was applied, was a more traditional method and was a high-manpower, low-efficiency task that required long labor. This is because each frame and each pixel was examined with the human eye and details that had to be fine-tuned were needed.

Upscaling market changed by AI

4by4, a visual tech solution company, said, “Before AI was introduced, we used a simple ‘open source filter’ to fill in the gaps between pixels. Workers worked on 30 still images one by one with each filter. “The work hours and labor intensity were long and strong,” he said.

In this way, the explanation is that the upscaling work in the past was a task that brought the degree of image improvement judged by human subjectivity to a level that was satisfactory to as many people as possible. Because it relied on human subjectivity, that is, human eyes, the work time was bound to be long.

However, with the introduction of AI, the efficiency of upscaling operations has increased exponentially. Fourbyfour, which provides an AI upscaling solution, explained the change after the introduction of AI technology, saying, “It took less than 3 hours for a job that would take more than two weeks for a person to do.” Forby4 is currently taking a ‘B2B2C’ approach to reach consumers through video service providers.

There are also Microsoft’s VSR and Nvidia’s RTX VSR as AI upscaling technologies that can be used by general consumers.

Microsoft’s (MS) Edge browser VSR

Microsoft (MS) recently introduced the Video Super Resolution (VSR) feature of the Edge browser. VSR is a technology that uses machine learning (ML) to improve the quality of any video you watch on (edge) browsers.

Microsoft’s VSR technology erases blurry compression artifacts and upscales the resolution of video played in the browser. VSR allows users to enjoy enhanced video content from video streaming platforms such as YouTube without sacrificing network bandwidth.

To use the Edge Browser VSR, you need an Nvidia RTX 20/30/40 series graphics card or an AMD RX 5700 to 7800 series graphics card. To use the VSR function, the video played in the browser must be played at a resolution of less than 720p and the device used must be connected to AC power.

The Edge browser VSR is available on Canary Channel, which tests the latest technology, and displays an HD icon in users’ address bar when the feature is activated. Click on the corresponding icon to activate or deactivate the corresponding function. Users can also manually activate the VSR.

Regarding the commercialization of the VSR function, a Microsoft official said, “Microsoft is improving ML (Machine Learning) model and hardware coverage to provide a better experience to more users. The timing of commercialization will be disclosed at a later date. Depending on whether or not, we expect that we will be able to share related information.”

Nvidia’s RTX VSR

Nvidia recently introduced the RTX Video Super Resolution (VSR) feature as part of a driver update to version 531.18. RTX VSR, which can be used on computers equipped with RTX 30/40 series graphics cards, upscales the video that users watch in a web browser with AI technology. Since RTX VSR operates on the Chromium framework, it can be used in web browsers such as Google Chrome and Microsoft Edge based on it.

In RTX VSR, bi-cubic upscaling, which is a conventional upscaling method, and upscaling through AI are performed together. It is a method of improving compression artifacts left by existing upscale images through images created by AI. Kim Seon-wook, technical marketing manager at Nvidia, explained the advantages of AI-based upscaling compared to other upscaling, saying, “The results of artificial intelligence (AI) are superior to programs created by humans based on algorithms.”

To use the RTX VSR, you must first launch the Nvidia Control Panel and open the ‘Adjust Video Image Settings’ window. Tick ​​the Super Resolution checkbox located under ‘RTX Video Enhancement’ allows you to choose from the lowest to highest upscaling quality. RTX VSR can be used with the latest GeForce Game Ready drivers or the latest Nvidia Studio drivers installed.

‘Free’ VSR Service vs ‘Paid’ Upscaling Software

In addition to Microsoft and Nvidia, there are various companies that provide AI upscaling services. The VSR services introduced above upscale the video or image displayed in the web browser when the corresponding function is activated. This method has the advantage that the user can easily and quickly access the upscaling function, but also has the disadvantage that the user cannot directly participate in post-processing.

By purchasing upscaling software, available for a fee from several companies, such as Topazlabs, users have more processing options during the upscaling process. Since the degree of ‘improvement’ of an image is ultimately determined by the subjectivity of the user, having more options for the user in the image processing process is an important factor in upscaling. However, since there is a difference in time and cost when comparing these paid upscaling software and VSR service, consumers will have to think about which upscaling service to use by weighing the benefits and losses.

Reporter Lee Sang-hoon [email protected]