How waifu2x can save your macOS retouching experience by "taking the veil off" your images with AI
We need pictures, clear pictures.
From traditional media to online communities, images have always been among the fastest and most visual information carriers, no matter how much the way of creation has changed. Although there are countless ways to search for images on the Internet today, it's not always easy to get access to high quality images outside of dedicated stock sites. In the past, we used "search engines + keywords" to search for fishing material in a wide net; when artificial intelligence started to emerge, using "image search" to trace the source of images became the main way to get high-definition material.
Nowadays, it is not only possible to "HD" images directly using AI technology, but it has also developed into a simple and efficient image restoration technique. Especially with the help of some open source tools and models, ordinary users can also use this technique to restore images and videos on their phones and computers. waifu2x is one of the well-reputed and well-maintained options.
Finally, a developer has brought it to macOS.
What is waifu2x
Convolutional Neural Networks (CNN) is a class of deep neural networks commonly used in visual image analysis. Since its convolutional layer splits and filters the features learned by the machine, even a certain degree of displacement and deformation of the input object does not affect the resultant output.
waifu2x is an image processing algorithm that uses deep convolutional neural networks to restore and scale images or videos in high definition. Compared with traditional interpolation algorithms, waifu2x's Super-resolution imaging (Super-resolution) + noise reduction mechanism can avoid jaggies, blurring, and color blocks to the maximum extent, and improve the sharpness and purity of the image, thus achieving improved visual perception.
Initially, waifu2x was mainly used to repair ACG content. With the widespread adoption of this technology and the popularity and depth of machine learning techniques, various waifu2x GUIs developed by third-party developers gradually became mainstream, and many waifu2x-based models were trained for application to specific content images. As a result, waifu2x has greatly improved its ubiquity and significantly lowered the threshold of operation, and there have been many easy-to-use waifu2x software on Windows platform, but for various reasons of software adaptation and hardware compatibility, the situation on the macOS side has been somewhat lackluster.
The waifu2x software available for macOS has long been flawed in function, performance, and stability to one degree or another, but these slightly flawed creations are generally the work of enthusiasts, and there's no excuse for the problems. Well, GitHub's independent developer @Vaida has brought us a much better waifu2xExtension.
waifu2xExtension is easy to use and powerful. After following this article and completing the basic configuration, we can use waifu2x for image processing on macOS elegantly.
*The demo content in this article is from version 5.0 Beta 8 and cannot be guaranteed to be consistent in operation and functionality with future updated software versions.
algorithm model
waifu2xExtension comes with the algorithm model waifu2x-caffe, which can be installed directly and works fine. However, if you need to process more complex images and get the best results, you must install a specific algorithm model. waifu2xExtension supports six algorithm models, which are.
- Real-CUGAN ncnn Vulkan - Custom AI super-resolution algorithms for ACG content
- Real-ESRGAN - AI super-resolution algorithm that favors ACG content and has generalizability (fast, less effective for faces and text content)
- RealSR ncnn Vulkan - AI super-resolution algorithm that favors real content
- CAIN ncnn Vulkan - AI video framing algorithm that can only be used for 0.5 time points (two frames interpolated into one frame)
- rife-ncnn-vulkan - AI video framing algorithm that can only be used for 0.5 time points (two frames interpolated) (faster and very good)
- DAIN ncnn Vulkan - AI video frame-completion algorithm with support for arbitrary point-in-time interpolation (slowest, highest footprint, very effective)
You can download algorithm models on demand and place them in the same fixed path, here it is recommended to create a folder directly in the Manuscript dedicated to models for subsequent updates. waifu2xExtension can also use algorithm models that are not in this list, but compatibility issues may arise.
Installation and use
After installing and opening waifu2xExtension on macOS, you can access the model management interface of waifu2xExtension in the software settings. Whether you are installing a model for the first time or updating it, just select "Open Finder" on the right side of the corresponding option, and then select the folder of the corresponding model in the pop-up access window.
As shown in the image, models that have been installed will display the installation status and file size to the left of the name and below, respectively, but You can right-click on the model that needs to be updated and select "Show on Github" in the menu to quickly jump to the release page and download and install the update manually.
In addition, waifu2xExtension supports TTA (Test Time Augmentation), a technique designed to process the input image with various transformations, including cropping different areas and changing the zoom level, to create several different versions, which are then compared to adjust the output for better image quality. . If your mac has poor performance, turning off this feature by checking "Disable TTA" in the settings can improve processing efficiency, software stability, and wait times to a certain extent.
With the addition of support for multiple models, waifu2xExtension adds a "Import - Process - Export" process to the normal waifu2x software, as well as processing and export settings (currently including image content, scaling, and noise reduction parameters, with more options to be added in the future. Processing - Export" process than the normal waifu2x software, adding the process of selecting models, as well as processing and export settings adapted to each model (currently including image content, scaling and noise reduction parameters, with more options to be added in the future).
Use the "Add" function, or drag the files or folders to be processed directly into the main window of waifu2xExtension and click "Done" in the upper right corner, the software will integrate the available models and options in the export screen for selection and adjustment, then click "Done" in the lower right corner of the export screen to process the current settings and export the files.
Summary
The waifu2xExtension is probably the best waifu2x front end for macOS at the moment, and its advantages consist mainly of the following.
- Support a variety of image and video formats, not only for image HD processing, but also for video framing
- Six additional optional algorithm models are supported to select the best quality for different styles of images by switching algorithm models
- All built with SwiftUI, which runs with great efficiency and stability on macOS
- Supports hardware acceleration by invoking ANE on M1/M2 macs, significantly outperforming traditional CPU/GPU acceleration in terms of processing time at the same power consumption
- Supports completely offline operation and privacy friendly
The latest waifu2xExtension 5.0 Beta 8 version has rewritten the software, some features of the old version are not yet live on the new version. If you need advanced features such as pre-export preview and model comparison, you can use the 4.1.3 stable version first, and then upgrade when the 5.0 version is fully developed.
colored egg
We used waifu2xExtension to restore the illustrations of fan submissions from the "Cooking Machine Network" period to a standard of "clearer than the original".
