Apps undertake Transformer to help extra dependable and performant media modifying use circumstances

Posted by Caren Chang – Developer Relations Engineer

The Jetpack Media3 library allows Android apps to construct top quality media apps. As a part of the Media3 library, the Transformer module goals to supply straightforward to make use of, dependable, and performant APIs for transcoding and modifying media.

For instance, apps can use Transformer to use modifying operations equivalent to trimming an extended piece of media file, or making use of results to video tracks. Transformer may also be used to transform media recordsdata from one format to a different, equivalent to adjusting the decision or encoding of the media file.

Growing Transformer APIs

As a part of the method to introduce new APIs, our engineering staff works intently with Google apps equivalent to Google Images to check and experiment the brand new APIs. Experimental flags are first launched to allow efficiency enhancements. As soon as the outcomes are profitable and conclusive, these experimental options are then constructed into the default API implementations or promoted to public APIs for all apps to make use of. This method permits Transformer APIs to be examined on all kinds of gadgets.

Transformer Adoption in apps

Apps which were utilizing Transformer in manufacturing noticed in-app efficiency enhancements, much less code to take care of, and higher developer expertise. Let’s take a better have a look at how Transformer has helped apps for his or her media-editing use circumstances.

One in every of customers’ favourite options in Google Images is reminiscence sharing, the place snippets of your life story which are curated and offered as Google Images recollections can now be shared as movies to social media and chat apps. Nonetheless, the method of mixing media objects to create a video on system is useful resource intensive and topic to important latency, particularly on low-end gadgets. To cut back this latency and allow the function on a wider vary of gadgets, Images adopted Transformer of their media creation pipeline. Together with different enhancements made, the staff discovered that Transformer performed an element in decreasing the median consumer latency for creating reminiscence movies by 41% on high-end gadgets and 27% on mid-range gadgets.

The Images app additionally allows customers to carry out media edits equivalent to trimming or rotating a video. By adopting Transformer APIs for rotating movies, median save latency was diminished by 79% for relevant movies. The app additionally adopted Transformer’s API for optimizing video trimming, and noticed video save latency lower by 64%.

1 Second On a regular basis is a private video journal that helps you create fascinating montages and timelapses. One of many app’s important consumer journeys is sequentially combining quick movies to create a significant film. After adopting Transformer for this use case, the app noticed that video encoding efficiency was as much as 5x quicker, permitting them to discover enabling 4k and HDR help. The Transformer adoption additionally helped lower related code by 30%, making it simpler for the builders to take care of the code base.

BandLab is the next-generation music creation platform utilized by hundreds of thousands around the globe to make and share their music. The app initially used MediaCodecs for his or her video creation use circumstances, however discovered that the low stage implementation resulted in native crashes that had been tough to debug. After researching extra on Transformer, the staff made the choice emigrate from MediaCodecs to Transformer. General, it solely took the staff 12 working days for the migration, and this resulted in an easier codebase and extra maintainable pipeline for his or her media creation use circumstances. As well as, the app noticed that each one beforehand noticed native crashes had been now not occurring anymore.

What’s subsequent for Transformers?

We’re excited to see Transformer’s adoption within the developer group, and can proceed including new options to help extra media-editing use circumstances for the Android ecosystem together with:

• Higher help for previewing media edits
• Enhancing the efficiency and developer expertise for video body extraction
• Simpler integration with AI results
• and far more

Maintain a watch out on what we’re engaged on within the Media3 Github, and file function requests to assist form the way forward for Transformer!