In order to let you know the latest progress of the company in artificial intelligence, Microsoft today announced a new investment in power apps rapid design tool As a new feature in its codeless / low code power apps service set, express design can convert uploaded PDF, PowerPoint, figma design files, screenshots (or hand drawn sketches) into a working application (requiring a series of tricks).
(from: Microsoft blog)
Microsoft said at the build 2022 developer conference that express design uses the industry-leading artificial intelligence technology to scan input and generate working application controls that support data storage.
As the main AI driving function after power FX, it uses openai's gpt-3 scheme to intelligently translate instructions such as "find products whose names begin with 'child' into formulas".
Obviously, Microsoft wants to cater to the increasingly popular trend of codeless / low code programming. Gartner points out that by 2025, up to 70% of applications will be developed with this scheme (including AI technology).
A further prediction is that the proportion of ordinary "citizen developers" who lack a formal computer science background in large organizations is also expected to surpass traditional developers within this year.
Power apps express design example
In a series of blog posts, Microsoft explained the working principle of this rapid design tool - first perform image recognition with the object detection model of azure cognitive service platform, and then map the tables and graphics in documents (including sketches) to known application structures.
Express design understands buttons, groups, or text boxes and automatically generates working (or at least similar) applications based on these drawing elements. However, at this stage, this power apps rapid design tool can not help you complete all the arduous development work.
After scanning the file, you still need to manually mark application components such as fields, text labels, date selectors, and buttons and connect them to the database in the Microsoft dataverse service.
As with applications created in the normal way in power apps, users can add additional screens (manually or by converting new sketches) and upload images to customize backgrounds, buttons, and other components before publishing.
Turn a sketch into an app with express design in Power Apps（via）
Interestingly, as early as 2018, Microsoft has tried to "convert sketches into code" for the first time on the sketch2code project. The research project aims to use AI technology to convert handwritten drawings into working HTML prototypes.
However, in addition to Microsoft, many people have studied similar ideas. For example, airbnb and Alibaba have developed their own AI systems to convert the sketches on the interface into product models and codes.
In terms of business applications, a start-up called zecoda is using AI technology to automatically convert design documents into front-end code. However, Microsoft's express design scheme obviously has a larger scale advantage.
At present, express design is open to all power apps customers. Interested friends can move to the official portal to get more details (portal) 。