Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft: Add example and documentation for stand-alone apps with support for launching on vehicle client #158

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

DanielKuhn
Copy link
Contributor

CarPlay and Android Auto apps are expected to be able to get launched without the user having to open the respective app on the phone first - or even worse: have the phone app running at all times in order to use the CarPlay-app properly.

The example iOS-app featured in apps/example does not support launching on the CarPlay-client directly and is heavily intertwined with the phone app by presenting each CarPlay-template via a corresponding screen in the phone app and navigating to each screen when pushing the template onto the CarPlay-stack.

Since a lot of people who are using this package are wondering how to start their CarPlay-scene without having the app running in the background, I think there should be an example which illustrates this workflow and documents what's happening behind the scenes (pun intended).

This PR is an approach to addressing this issue by adding a simple CarPlay-only example app which can either get launched with the phone app open or, more importantly, WITHOUT opening the app on phone first.

Clone the original example as stand-alone-example
in order to demonstrate launching an app using this
package directly on the CarPlay-/AndroidAuto-client
without having the app running on the phone.

The stand-alone-example only renders a placeholder
text on phone and contains all it's CarPlay-logic
inside hooks which listen for
- CarPlay connection changes
- state changes (via useReducer)

The stand-alone-example CarPlay-app features
a TabTemplate as root template containing
a top-level ListTemplate with two browsable items.
Selecting a top-level item pushes a new
(sub-level) ListTemplate onto the CarPlay-stack,
which contains two non-browsable items.
Selecting a sub-level item presents a CarPlay
modal.
Do not create multiple bridges in PhoneScene.
Reuse the AppDelegates rootView and window.
When launching the CarPlay scene directly on
the CarPlay-client, create a bridge for the AppDelegate.
Again check for an already existing bridge in order to
not create multiple bridges:
If the app is already running on the phone, a bridge will
already be present on the AppDelegate.
If not: create one for the CarPlay scene.
This adds and uses an alternative approach to
initializing the app provided by @gavrichards:
- Do not call RCTAppDelegate's
application:didFinishLaunchingWithOptions
but instead cherry-pick the code from RCTAppDelegate's
application:didFinishLaunchingWithOptions
except for window and rootViewController creation
- move window and rootViewController creation to
PhoneScene since they're not needed in stand-alone
CarScene
- call initAppFromScene() both in PhoneScene and
CarScene to init the app.

This approach works for both startup scenarios,
both on Phone and on CarPlay-client.

Bonus:
The rootView property is stored in AppDelegate,
so it can be used in PhoneScene
(i.e. to pass to RNBootSplash if used)

Caveat:
The code in initAppFromScene() needs to
be adjusted to the specific version of react native
you are using!
The version used in stand-alone-example is
currently 0.71.13, so the code is taken from that
versions RCTAppDelegate and converted to Swift
(with a little help from ChatGPT for the C++ block,
so no guarantee that it works! I'm not on RN's
new architecture yet).
@DanielKuhn
Copy link
Contributor Author

DanielKuhn commented Nov 21, 2023

This is still a draft, since it covers only CarPlay for now. The Android part is simply a copy of the original example.
I will try to update the PR with a stand-alone, natively launchable Android Auto app as well. But for now this might help some people with iOS / CarPlay at least.

For a while I thought I'd be fine with the solution from @mitchdowney over at Podverse outlined in this PR (still present but commented out as "Approach 1" in the new stand-alone-example app) but after a while I found that it produced unpredictable and buggy results in combinations of phone app running/not running or starting on CarPlay first, then on phone, then killing phone again, etc...

Finally, after inspiration from @gavrichards ("Approach 2" in the new stand-alone-example app, initially outlined in this issue) and a lot of native debugging on the console of the physical device (since you don't have either Xcode nor React Native logs available when in CarPlay stand-alone mode!) I found this solution to work reliably in all circumstances / launch orders / lifecycle states.

Thanks to @gavrichards for the groundwork around initAppFromScene - and thanks to @birkir for maintaining this awesome package.

Feel free to comment and add improvements.

External display connection invokes
scene:willConnectTo:options again with the
session.role .windowExternalDisplayNonInteractive:
https://developer.apple.com/documentation/uikit/windows_and_screens/presenting_content_on_a_connected_display

This needs to be explicitly handled or rejected,
otherwise the app crashes with the error:
Terminating app due to uncaught exception 'UIViewControllerHierarchyInconsistency',
reason: 'A view can only be associated with at most one view controller at a time!"
@bitcrumb
Copy link

@DanielKuhn For what it is worth, I tried re-opening the discussion about compatibility of RN with scenes here.

@KestasVenslauskas
Copy link

Even this works & spawns the RN app but lots of apps have logic embeded inside components withing the App. So this is usefull only to show some "initial" termplate.
For example if I want to launch a player I can't because it's not rendered yet. Once I open the app it's fine.

@DanielKuhn
Copy link
Contributor Author

Even this works & spawns the RN app but lots of apps have logic embeded inside components withing the App. So this is usefull only to show some "initial" termplate. For example if I want to launch a player I can't because it's not rendered yet. Once I open the app it's fine.

I'm using this approach to render a "stand-alone" CarPlay-app. All logic for the app is encapsulated within a single component with hooks handling the CarPlay-events as outlined in the example

I'm also controlling react-native-track-player via these hooks in my production app.

@KestasVenslauskas
Copy link

@DanielKuhn Yes but this is not a real world example. In my case I use other player providers that are working only after render(). In this case I have to introduce react-native-track-player dependency and handle events with it untill the app is actually open. So my question is there a possability to make app call render and actually populate the component tree so the logic inside components would work?

@DanielKuhn
Copy link
Contributor Author

It's a very real world example with a couple 100k users 😄
When developing a CarPlay app the goal should always be that you don't need to have the phone at hand - that's the very purpose of CarPlay. Therefore rendering anything on the phone should not be necessary.
Maybe you'll find a way to implement the logic you need for your CarPlay app without the need to render anything in your phone app?
But even so: I don't see a reason why you couldn't attach app logic to the render method. After all the component in the example is rendered as well when the react native app is initialized in headless mode by starting it on CarPlay. All hooks are executed, all contexts are there... The only thing I found not working in the typescript code are setInterval and setTimeout - these are actually stopped when the phone goes into standby. As soon as I pick it up (screen lights up and the native player controls are shown on the lock screen) they start running again.

@KestasVenslauskas
Copy link

You are right about that app should not render anything. But I get weird results while app is started.
So the App component is loaded with hooks all good, but it should return other nested components that should be rendered or at least their render() method called. It looks like App render function is called but not nested components. I will try investigating a bit more later. Anyway thank you for the solution!

@alex-vasylchenko
Copy link

@DanielKuhn hi, have you already tried to update to react-native 0.74?
If you remember, my code is very similar to yours, but after the update everything broke.
I'd love to chat with you again, come to Discord

@KestasVenslauskas
Copy link

@DanielKuhn do you encounter app crash after re-connecting to carplay and clicking on any item that has onClick callback?
I do have your setup but I do experience #184 issues when re-connecting to carplay while single app instance is running.

@DanielKuhn
Copy link
Contributor Author

DanielKuhn commented Jun 10, 2024

@alex-vasylchenko I haven't updated to 0.74 yet, but as I mentioned on Discord: Every RN upgrade requires an adaptation of the AppDelegate's implementations of application:didFinishLaunchingWithOptions and initAppFromScene respectively to the new implementations of RCTAppDelegate.

@KestasVenslauskas I'm using version 2.3.0 still and I do observe blinking list elements occasionally, but only in the CarPlay Simulator, not on real devices. No crashes either.
BTW: From your screen recording I can see you're still using version 1 of "CarPlay Simulator.app" - try upgrading to version 2, available in the Downloads section of Apple's developer resources. It has some nice new features like customizable Widescreen Configs available from the system tray.

@mefjuu
Copy link

mefjuu commented Jun 27, 2024

@DanielKuhn Did you manage to run the app from Android Auto directly (without running the app on mobile device first)? There is nothing but "RNCarPlay loading..." screen in such scenario.

@DanielKuhn
Copy link
Contributor Author

@DanielKuhn Did you manage to run the app from Android Auto directly (without running the app on mobile device first)? There is nothing but "RNCarPlay loading..." screen in such scenario.

I'm not an Android developer, therefore I cannot say whether it is even possible to start a react native app in headless mode directly from Android Auto. Actually that's the main reason this pull request is still a draft.

What we did in our production app was to require the "draw over apps"-permission in Android, so that the app can be started while the phone is actually locked. This way, while not headless like in iOS, the app can still handle all events coming from Android Auto.

@mefjuu
Copy link

mefjuu commented Jul 26, 2024

Has anyone applied these changes to RN 0.74.x-based project?

@gavrichards
Copy link

I am also looking for an example of these changes for RN 0.74, due to the changes they've made to application:didFinishLaunchingWithOptions. Some of the methods they call now aren't accessible from our own AppDelegate. Also createBridge now returns nil regardless.

@KestasVenslauskas
Copy link

Has anyone experienced issues with iOS 17.6 when the app starts only when phone screen turns on? It's like it's blocked from initializing until user touches the phone...

@DanielKuhn
Copy link
Contributor Author

@gavrichards
It looks like bridge creation is now implemented via the new RCTRootViewFactory.
I'm by far not a Swift/ObjC developer, but from reading the code it looks like the RCTAppDelegate creates a RCTRootViewFactory inside didFinishLaunchingWithOptions which creates a weak reference to the RCTAppDelegate itself and forwards createRootViewWithBridge and createBridgeWithDelegate to this reference...
This is all done in createRCTRootViewFactory.

I tried recreating this method inside our custom AppDelegate but did not succeed. What works though is exposing createRCTRootViewFactory in RCTAppDelegate.h. This way we can call it from our custom AppDelegate and voilá - a working bridge is created.

I upgraded my standalone-example to RN 0.75 in this branch. It is based on this PR which upgrades the regular example to RN 0.75 first.

Check out the call to createRCTRootViewFactory() in AppDelegate which is possible via this patch.

Let me know what you think of this (ugly but working) solution.

@gavrichards
Copy link

gavrichards commented Aug 25, 2024

@DanielKuhn this is excellent, thank you so much for sharing your solution. I'm so glad there's someone else out there trying to do the same niche things I am, I'd be pretty stuck otherwise!

One bit I'm struggling with when trying to replicate your solution is the introduction of "RCTColorSpaceUtils".
I've added this to the Bridging Header, but where it's referenced in AppDelegate I'm getting an error:

CleanShot 2024-08-25 at 15 03 18@2x

Any idea what might be going on here?

EDIT: oh - is that a 0.75 specific thing? I'm still only trying to update to 0.74 at the moment.

@DanielKuhn
Copy link
Contributor Author

DanielKuhn commented Aug 25, 2024

@gavrichards yes, it was introduced in 0.75.
Discussion: react-native-community/discussions-and-proposals#738
PR: facebook/react-native#42830

The root view factory approach should work in both 0.74 and 0.75.

@DanielKuhn
Copy link
Contributor Author

I opened a PR requesting to expose createRCTRootViewFactory in RCTAppDelegate.h: facebook/react-native#46211
Let's see what they say.

@DanielKuhn
Copy link
Contributor Author

@gavrichards The rn maintainers say we should instantiate a factory ourselves: facebook/react-native#46211
For me the approach outlined in facebook/react-native#46184 (comment) is not working, though: When I instantiate a factory in initAppFromScene, the app runs fine (PhoneScene) but CarPlay (CarScene) stays black.
I'll stick with the patch for now, let me know if you figure out how to use the factory without it.

@KestasVenslauskas
Copy link

Still no luck running this on new arch?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants