[feat] iOS Widget support

[Tehnix]

Describe the problem

With Tauri 2.0 hitting beta and bringing Mobile support, one key part of the Mobile (and Desktop) ecosystem is still missing: Widgets.

Right now I'm unaware of any way to write Widgets in Tauri, or even combine an Tauri-based App with e.g. a Swift-based Widget.

E.g. something like this:

Describe the solution you'd like

I would love to see way for Tauri to support Widgets on Mobile and Desktop. My primary focus is personally on iOS and macOS Widgets, but Android Widgets should also be considered, although I would think these are different enough that it might make sense to treat them as completely separate.

While we cannot use Webviews in Widgets, I could imagine a way similar to https://scriptable.app where Tauri exposes an API and a Widget runtime.

• The Widget API: Configuration of Layout for the View and data loading that should get run in getSnapshot/getTimeline of the Widget (iOS and macOS is the same)
• The Widget Runtime: Native Swift/Kotlin code that sets up the plumbing and executes what the user has set up in getSnapshot/getTimeline and the view etc

I'm not entirely sure how Scriptable does it, but it seems to expose a minimal set of the Widget API via JavaScript, which lets you do most things. It hasn't been updated to support interactivity yet though (arrived in iOS 17).

Alternatives considered

A different approach one could imagine would be a way to link your Widget code from an existing Xcode project together with the Tauri generated Xcode project.

Additional context

No response

[AravindPrabhs]

I have recently experimented with this and had partial success with communicating with a iOS SwiftUI live activity widget. The steps I followed was:

1. Open the generated Xcode project and adds a widget extension (with live activity) as well links it the main app. This appropriately adds the new widget target that will be made correctly when xcodebuild is called by tauri ios.
2. Create a Swift package that contains the attributes of the widget. Modify your extension to use these attributes, this is all the linking between widget extension that was needed, ActivityKit takes care of pulling up the correct widget/live activity.
3. Add swift-rs to your project and link to your Swift package in build.rs - I created an iOS only package so only linking it iOS.
4. Create public methods that create, update and close your live activity.
5. Use swift! to link to these methods.

Curiously, I was not able to get it working in dev mode but when I did tauri ios build it was fine.

The swift package was roughly like this:

import Foundation
import ActivityKit
import SwiftRs


// I previously tried sending this across with the Swift-Rust boundary
// so made it an NSObject but it can be a struct as normal
public class LiveActivityAttributes: NSObject, ActivityAttributes {
    internal init(name: String) {
        self.name = name
    }
    
    public class ContentState: NSObject, Codable {
        internal init(emoji: String) {
            self.emoji = emoji
        }
        
        // Dynamic stateful properties about your activity go here!
        public var emoji: String
    }

    // Fixed non-changing properties about your activity go here!
    public var name: String
}

// This is how we keep track of the activity and must be an NSObject
// as this goes over the Swift-Rust boundary
public class LiveActivityHandle: NSObject {
    internal init(activity: Activity<LiveActivityAttributes>) {
        self.activity = activity
    }
    
    var activity: Activity<LiveActivityAttributes>
}

// the public facing methods we link to
// they need to be SwiftRs compatible to
// extends this emoji will probably become fully fledged struct but
// will need to be an NSObject to go across the boundary

@_cdecl("start_live_activity")
public func startLiveActivity() -> LiveActivityHandle? {
    print("requesting activity");
    if ActivityAuthorizationInfo().areActivitiesEnabled {
        do {
            let static_attributes = LiveActivityAttributes(name: "Aravind")
            let initial_state = LiveActivityAttributes.ContentState(
                emoji: "😀"
            )
            
            let activity = try Activity.request(
                attributes: static_attributes,
                content: .init(state: initial_state, staleDate: nil),
                pushType: nil
            )
            
            return LiveActivityHandle(activity: activity)
        } catch {
            print(
                """
                Couldn't start activity
                ------------------------
                \(String(describing: error))
                """
            )
        }
    }
    print("failed to get activity");
    return nil
}

@_cdecl("update_live_activity")
public func updateLive(handle: LiveActivityHandle, emoji: SRString) {
    Task.init {
        await handle.activity.update(
            ActivityContent<LiveActivityAttributes.ContentState>(
                state: LiveActivityAttributes.ContentState(emoji: emoji.toString()), 
                staleDate: nil,
                relevanceScore: 100
            ),
            alertConfiguration: nil
        )
    }
}

@_cdecl("close_live_activity")
public func closeLive(handle: LiveActivityHandle, emoji: SRString) {
    Task.init {
        await handle.activity.end(ActivityContent<LiveActivityAttributes.ContentState>(
            state: LiveActivityAttributes.ContentState(emoji: emoji.toString()),
            staleDate: Date.now,
            relevanceScore: 100
        ), dismissalPolicy: .default);
    }
}

The rust code that called it for .setup:

use std::ffi::c_void;

// most definitely a better way to do the following but was want
// a swift_rs::SwiftObject where we don't care about the data
#[repr(C)]
#[derive(Clone)]
struct ActivityHandle(*const c_void);
// SAFETY: we only have one async block handling the instance
unsafe impl Send for ActivityHandle {}
unsafe impl Sync for ActivityHandle {}
impl Copy for ActivityHandle {}
impl ActivityHandle {
    fn is_some(&self) -> bool {
        !self.0.is_null()
    }
}
impl<'a> swift_rs::SwiftArg<'a> for ActivityHandle {
    type ArgType = *const c_void;

    unsafe fn as_arg(&'a self) -> Self::ArgType {
        self.0
    }
}
impl swift_rs::SwiftRet for ActivityHandle {
    unsafe fn retain(&self) {
        retain_object(self.0);
    }
}

// link to swift methods
swift_rs::swift!(pub(crate) fn retain_object(obj: *const c_void));
swift_rs::swift!(pub(crate) fn release_object(obj: *const c_void));
swift_rs::swift!(fn start_live_activity() -> ActivityHandle);
swift_rs::swift!(fn update_live_activity(handle: ActivityHandle, emoji: swift_rs::SRString));
swift_rs::swift!(fn close_live_activity(handle: ActivityHandle, emoji: swift_rs::SRString));

// start activity on main thread
let activity = start_live_activity();
if activity.is_some() {
  tauri::async_runtime::spawn(async move {
      
      // send updates from potentially background threads
      let random_emojis = ["🚢","🙇","😮","💝","🛠","👗","🌦","🆖","🏖"];
      for emoji in random_emojis {
          tokio::time::sleep(Duration::from_millis(500)).await;  
          update_live_activity(activity, emoji.into());              
      }
      close_live_activity(activity, "🏁".into());
      // manually drop the activity object !
      release_object(activity.0);
  });
}

Overall it feels tricky to generalise. Maybe a CLI lead approach that can generate the Swift package that matches a Rust struct that represents the attributes. It could also add the widget extension that uses these attributes and links it appropriately. Also still unsure why it doesn't work in dev mode. I just get a white screen with the WebView failing to load.

[AravindPrabhs]

The white screen was just a problem with my network and connecting the dev server so not a issue from the widget stuff at all.

[sanabel-al-firdaws]

im currently making a pwa and i am trying to find a way to make widgets on andriod and ios

currently on windows i can make widgets with my pwa its a new feature they added it would be nice if i can do the same on andriod

would tauri be able to do it?