Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VideoFrameDecoder to support decoding video frames from any source. #689

Merged
merged 8 commits into from
Mar 4, 2021

Conversation

colinrtwhite
Copy link
Member

Fixes #687.

This finally adds support for decoding video frame from any source (including the network!) with an important caveat. VideoFrameDecoder creates a short lived file on disk to buffer the source into. This is less performant than decoding the file in memory and it requires enough space on disk to copy the video which could be a problem for very large video files.

VideoFrameFileFetcher and VideoFrameUriFetcher don't have this caveat which is why they're not deprecated. Long term (possibly in the next major version) we'll hopefully be able to remove this limitation on VideoFrameDecoder and deprecate the fetchers.

import coil.size.Size
import kotlin.math.roundToInt

internal class VideoFrameDecoderDelegate(private val context: Context) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is basically pulled out entirely from VideoFrameFetcher without changes.

size: Size,
options: Options
): DecodeResult {
val tempFile = File.createTempFile("tmp", null, context.cacheDir.apply { mkdirs() })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need write permissions to do this, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! The cacheDir is free to write to.

Copy link
Member

@Jawnnypoo Jawnnypoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Great work, I know this is much appreciated from those who need it.

@colinrtwhite colinrtwhite merged commit 18fa89a into master Mar 4, 2021
@colinrtwhite colinrtwhite deleted the colin/video_frame_decoder branch March 4, 2021 19:09
@ultraon
Copy link

ultraon commented Apr 21, 2021

It doesn't look like a workable solution in conjunction with streaming content urls (mpd - MPEG DASH, m3u8 - HLS), right?

@colinrtwhite
Copy link
Member Author

colinrtwhite commented Apr 21, 2021

@ultraon It should work with any format supported by Android as long as the stream is not live and has completed. If you need video frames while the stream is in process I'd use ExoPlayer.

@yehia2030
Copy link

yehia2030 commented Feb 8, 2022

it still didn't work with me in compose and this is my example

        val context = LocalContext.current
        Image(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            painter = rememberAsyncImagePainter(ImageRequest.Builder(LocalContext.current)
                .data(data = "http:https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                .apply(block = {
                 decoderFactory(VideoFrameDecoder.Factory())
                }).build()),
            contentDescription = "test",
        )

or this code

        AsyncImage(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            model = ImageRequest.Builder(context)
                .data("http:https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
                .apply {  decoderFactory(VideoFrameDecoder.Factory()) }
                .build(),
            contentDescription = null
        )

if there is any solution please let me know

@yehia2030
Copy link

yehia2030 commented Feb 8, 2022

after a lot of pain the solution is

        val imageLoader = ImageLoader.Builder(context).build()
        val model = ImageRequest.Builder(LocalContext.current)
            .data(data = "http:https://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")
            .apply(block = {
                decoderFactory(VideoFrameDecoder.Factory())
                videoFrameMillis(50)
                listener(
                    onStart = {println("Start")},
                    onError = { _, result -> println("Error ${result.throwable.message}")},
                    onSuccess = { _, _ ->  println("Success")},
                    onCancel = { println("Cancel") }
                )
                size(100)
            }).build()
        Image(
            modifier = Modifier
                .fillMaxWidth()
                .height(200.dp)
                .padding(horizontal = 10.dp),
            painter = rememberAsyncImagePainter(model, imageLoader = imageLoader,
                onError = {
                println("Error 2 ${it.result.throwable.message}")
            }, onLoading = {
                    println("Loading 2 ")
                }, onSuccess = {
                    println("Success 2${it.result.isSampled} ")

                }),
            contentDescription = "test"
        )
        LaunchedEffect(key1 = true) {
            imageLoader.execute(model)
        }

@yehia2030
Copy link

could we make a buffer of something like that as if the video is long it takes too much time to load the image ? as i think it loaded the fully video then get its thumbnail

colinrtwhite added a commit that referenced this pull request Oct 5, 2022
…e. (#689)

* Add VideoFrameDecoder.

* Ensure directory exists.

* Updates.

* Catch IOException.

* Docs.

* Add a test.

* Update Application.

* Docs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support loading video frames from the network.
4 participants