Hello,
I've been tinkering a bit with TextComponent.
Based on the docs it seems like this component should always render sharp and nice text, no matter how close the user gets:
RealityKit dynamically adjusts the backing size to a value that results in high-fidelity text at its current location.
And it does on visionOS, but on iOS and macOS the text gets pixelated when I get close to it, as if its just rendering it once as a plain image texture.
Can anyone tell me if this is expected behavior or a bug?
Here two screenshots for comparison (iPhone and Vision Pro):
Thanks!
General
RSS for tagDiscuss Spatial Computing on Apple Platforms.
Selecting any option will automatically load the page
Post
Replies
Boosts
Views
Activity
Hello,
Let me ask you a question about Apple Immersive Video.
https://www.apple.com/newsroom/2024/07/new-apple-immersive-video-series-and-films-premiere-on-vision-pro/
I am currently considering implementing a feature to play Apple Immersive Video as a background scene in the app I developed, using 3DCG-created content converted into Apple Immersive Video format.
First, I would like to know if it is possible to integrate Apple Immersive Video into an app.
Could you provide information about the required software and the integration process for incorporating Apple Immersive Video into an app?
It would be great if you could also share any helpful website resources.
I am considering creating Apple Immersive Video content and would like to know about the necessary equipment and software for producing both live-action footage and 3DCG animation videos.
As I mentioned earlier, I’m planning to play Apple Immersive Video as a background in the app. In doing so, I would also like to place some 3D models as RealityKit entities and spatial audio elements.
I’m also planning to develop the visionOS app as a Full Space Mixed experience. Is it possible to have an immersive viewing experience with Apple Immersive Video in Full Space Mixed mode? Does Apple Immersive Video support Full Space Mixed?
I’ve asked several questions, and that’s all for now. Thank you in advance!
I've tried following apple's documentation to apply a video material on a Model Entity, but I have encountered a compile error while attempting to specify the Spatial Audio type.
It is a 360 video on a Sphere which plays just fine, but the audio is too quiet compared to the volume I get when I preview the video on Xcode. So I tried tried to configure audio playback mode on the material but it gives me a compile error:
"audioInputMode' is unavailable in visionOS
audioInputMode' has been explicitly marked unavailable here
RealityFoundation.VideoPlaybackController.audioInputMode)"
https://developer.apple.com/documentation/realitykit/videomaterial/
Code:
let player = AVPlayer(url: url)
// Instantiate and configure the video material.
let material = VideoMaterial(avPlayer: player)
// Configure audio playback mode.
material.controller.audioInputMode = .spatial // this line won’t compile.
VisionOS 2.4, Xcode 16.4, also tried Xcode 26 beta 2.
The videos are HEVC MPEG-4 codecs.
Is there any other way to do this, or is there a workaround available?
Thank you.
I posted https://developer.apple.com/forums/thread/809481 yesterday about an issue I discovered with pushWindow in visionOS 26.2 RC, but today I discovered a second problem with pushWindow.
If window A calls pushWindow to present window B, and the user pins window B to a wall, the following unexpected behaviors are observed:
Window B spontaneously disappears.
If the user re-launches the (still running) app from the visionOS home view, both window A and window B appear simultaneously. I assume only window B should be visible at this point, since window A pushed window B.
If the user closes window B, it's now impossible to present window B again. Calls to pushWindow appear to be ignored.
If the user force-quits the app and relaunches it, and pushWindow is called again, window B appears, but window A remains visible.
I also noticed this surprising behavior:
This broken state of pushWindow behavior now affects all other apps on the system that may call pushWindow in the future, not just the app whose pushed window was pinned above.
A workaround is to reboot the device, and then the system will behave as expected until the next time the user pins a pushed window.
Hi!
I'm currently trying to render another XR scene in front of a RealityKit one.
Actually, I'm anchoring a plane to the head with a shader to display for left/right eye side-by-side images. By default, the camera has a near plane so I can directly draw at z=0.
Is there a way to change the camera near plane? Or maybe there is a better solution to overlay image/texture for left/right eyes?
Ideally, I would layer some kind of CompositorLayer on RealityKit, but that's sadly not possible from what I know.
Thanks in advance and have a good day!
Hi ,
I'm struggling with visionOS window management and need help with closing child windows programmatically.
App Structure
My app has a Main-Sub window hierarchy:
AWindow (Home/Main)
BWindow (Main feature window)
CWindow (Tool window - child of BWindow)
Navigation flow:
AWindow → BWindow (switch, 1 window on screen)
BWindow → CWindow (opens child, 2 windows on screen)
I want BWindow and CWindow to be separate movable windows (not sheet/popover) so users can position them independently in space.
The Problem
CWindow doesn't close when BWindow closes by tapping the X button below the app (next to the window bar)
User clicks X on BWindow → BWindow closes but CWindow remains
CWindow becomes orphaned on screen
Can close CWindow programmatically when switching BWindow back to AWindow
App launch issue
After closing both windows, CWindow is remembered as last window
Reopening app shows only CWindow instead of BWindow
User gets stuck in CWindow with no way back to BWindow
I've Tried Environment dismissWindow in cleanup but its not working.
// In BWindow.swift
.onDisappear {
if windowManager.isWindowOpen("cWindow") {
dismissWindow(id: "cWindow")
}
}
My App Structure Code Now
// in MyNameApp.swift
@main
struct MyNameApp: App {
var body: some Scene {
WindowGroup(id: "aWindow") {
AWindow()
}
WindowGroup(id: "bWindow") {
BWindow()
}
WindowGroup(id: "cWindow") {
CWindow()
}
}
}
// WindowStateManager.swift
class WindowStateManager: ObservableObject {
static let shared = WindowStateManager()
@Published private var openWindows: Set<String> = []
@Published private var windowDependencies: [String: String] = [:]
private init() {}
func markWindowAsOpen(_ id: String) {
markWindowAsOpen(id, parent: nil)
}
func markWindowAsClosed(_ id: String) {
openWindows.remove(id)
windowDependencies[id] = nil
}
func isWindowOpen(_ id: String) -> Bool {
let isOpen = openWindows.contains(id)
return isOpen
}
func markWindowAsOpen(_ id: String, parent: String? = nil) {
openWindows.insert(id)
if let parentId = parent {
windowDependencies[id] = parentId
}
}
func getParentWindow(of childId: String) -> String? {
let parent = windowDependencies[childId]
return parent
}
func getChildWindows(of parentId: String) -> [String] {
let children = windowDependencies.compactMap { key, value in
value == parentId ? key : nil
}
return children
}
func setNextWindowParent(_ parentId: String) {
UserDefaults.standard.set(parentId, forKey: "nextWindowParent")
}
func getAndClearNextWindowParent() -> String? {
let parent = UserDefaults.standard.string(forKey: "nextWindowParent")
UserDefaults.standard.removeObject(forKey: "nextWindowParent")
return parent
}
func forceCloseChildWindows(of parentId: String) {
let children = getChildWindows(of: parentId)
for child in children {
markWindowAsClosed(child)
NotificationCenter.default.post(
name: Notification.Name("ForceCloseWindow"),
object: nil,
userInfo: ["windowId": child]
)
forceCloseChildWindows(of: child)
}
}
func hasMainWindowOpen() -> Bool {
let mainWindows = ["main", "bWindow"]
return mainWindows.contains { isWindowOpen($0) }
}
func cleanupOrphanWindows() {
for (child, parent) in windowDependencies {
if isWindowOpen(child) && !isWindowOpen(parent) {
NotificationCenter.default.post(
name: Notification.Name("ForceCloseWindow"),
object: nil,
userInfo: ["windowId": child]
)
markWindowAsClosed(child)
}
}
}
}
// BWindow.swift
struct BWindow: View {
@Environment(\.dismissWindow) private var dismissWindow
@ObservedObject private var windowManager = WindowStateManager.shared
var body: some View {
VStack {
Button("Open C Window") {
windowManager.setNextWindowParent("bWindow")
openWindow(id: "cWindow")
}
}
.onAppear {
windowManager.markWindowAsOpen("bWindow")
}
.onDisappear {
windowManager.markWindowAsClosed("bWindow")
windowManager.forceCloseChildWindows(of: "bWindow")
}
.onChange(of: scenePhase) { oldValue, newValue in
if newValue == .background || newValue == .inactive {
windowManager.forceCloseChildWindows(of: "bWindow")
}
}
}
}
// CWindow.swift
import SwiftUI
struct cWindow: View {
@ObservedObject private var windowManager = WindowStateManager.shared
@State private var shouldClose = false
var body: some View {
// Content
}
.onDisappear {
windowManager.markWindowAsClosed("cWindow")
NotificationCenter.default.removeObserver(
self,
name: Notification.Name("ForceCloseWindow"),
object: nil
)
}
.onChange(of: scenePhase) { oldValue, newValue in
if newValue == .background {
}
}
.onAppear {
let parent = windowManager.getAndClearNextWindowParent()
windowManager.markWindowAsOpen("cWindow", parent: parent)
NotificationCenter.default.addObserver(
forName: Notification.Name("ForceCloseWindow"),
object: nil, queue: .main) { notification in
if let windowId = notification.userInfo?["windowId"] as? String, windowId == "cWindow" {
shouldClose = true
}
}
}
.onChange(of: shouldClose) { _, newValue in
if newValue {
dismissWindow()
}
}
}
The logs show everything executes correctly, but CWindow remains visible on screen.
Questions
Why doesn't dismissWindow(id:) work in cleanup scenarios?
Is there a proper way to create a window relationships like parent-child relationships in visionOS?
How can I ensure main windows open on app launch instead of tool windows?
What's the recommended pattern for dependent windows in visionOS?
Environment: Xcode 16.2, visionOS 2.0, SwiftUI
Hey all,
I'm working on a visionOS app that captures live frames from the left and right cameras of Apple Vision Pro using cameraFrame.sample(for: .left/.right).
Apple provides documentation on encoding side-by-side frames into MV-HEVC spatial video using CMTaggedBuffer:
Converting Side-by-Side 3D Video to MV-HEVC
My question:
Is there any way to render tagged frames (e.g. CMTaggedBuffer with .stereoView(.leftEye/.rightEye)) live, directly to a surface in RealityKit or Metal, without saving them to a file?
I’d like to create a true stereoscopic (spatial) live video preview, not just render two images side-by-side.
Any advice or insights would be greatly appreciated!
I've been struggling with this for far too long so I've decided to finally come here and see if anyone can point me to the documentation that I'm missing. I'm sure it's something so simple but I just can't figure it out.
I can SharePlay our test app with my brother (device to device) but when I open a volumetric window, it says "not shared" under it. I assume this will likely fix the video sharing problem we have as well. Everything else works so smooth but SharePlay has just been such a struggle for me. It's the last piece to the puzzle before we can put it on the App Store.
In WWDC25 session What’s new for the spatial web, the presenter showed creating an immersive environment for a web page by adding to the page's HEAD section
<link rel="spatial-backdrop" href="office.usdz" environmentmap="lighting.hdr">
My first attempt failed, and I am trying to track down why.
Before I search all the potential failure paths, I wanted to ask the community,
Is this feature available in the latest visionOS 26 beta?
I haven't seen anyone talk about their use of the feature yet.
Topic:
Spatial Computing
SubTopic:
General
Hello, I was wondering how I can initialize an ImageAnchoringSource using
https://developer.apple.com/documentation/realitykit/anchoringcomponent/imageanchoringsource/init(_:)
When I construct one using a URL, it doesn't seem to be tracked and I see in the following when I debug print the component:
▿ 0 : AnchoringComponent
▿ target : Target
▿ referenceImage : 1 element
▿ from : ImageAnchoringSource
▿ url : Optional<URL>
▿ some : file:///var/mobile/Containers/Data/Application/D1126EA0-A1D7-468F-A40C-8578B7F5BDDF/Library/Caches/CodeCache/0E457AA7-2195-48B9-9DD4-58CEB9397F69.png
- _url : file:///var/mobile/Containers/Data/Application/D1126EA0-A1D7-468F-A40C-8578B7F5BDDF/Library/Caches/CodeCache/0E457AA7-2195-48B9-9DD4-58CEB9397F69.png
- _parseInfo : nil
- _baseParseInfo : nil
- name : nil
- group : nil
▿ trackingMode : TrackingMode
- trackingMode : 2
Is there a specific format for the parseInfo?
When I use the same image to make an image anchoring source by group and name in AR Resources, it is tracked.
Thank you!
I am trying to launch a fully immersive game from Unity on a SwiftUI view. The game is using Metal Rendering with Compositor Services.
I added the unity Xcode project into the workspace, added the necessary bridge code. When I click on the button to call ufw?.showUnityWindow(), it does not start and I get the following in the console:
AR session failed to start after 5 seconds. Is the app configured to use an immersive space?
We can add ornaments to popovers shown by PresentationComponent, but I’m not sure if we should.
While working on the editor for entities in a Volume-based app, I had the idea to add ornaments to the presented views. The entire app exists inside a volume. A user can tap a item to present a popoverUI to edit it. This is displayed using the new PresentationComponent in visionOS 26.
Ornaments have a new attachment anchor option this year: .parent().
.ornament(attachmentAnchor: .parent(.top), ornament: {...})
This works well in the Simulator. We can add ornaments around this popover view just like we would with a window.
Unfortunately, when I run this on device I get a different experience. Any part of the ornament that overlaps with the popover content isn’t rendered correctly. Sometimes it entirely disappears, other times it becomes partially transparent.
We could use content alignment to try to make sure the ornament doesn’t overlap the popover content.
.ornament(attachmentAnchor: .parent(.top), contentAlignment: .bottom, ornament: {...})
This works sometimes–but not all the time. It’s not clear if this is a bug or not, because I’m not sure if we are even supposed to be able to use ornaments in this way. Here is my hierarchy:
An app opens as a Volume
Volume presenting a RealityView, with its own ornament using .scene() anchor
Multiple Entities with Presentation Component show an edit view
The view uses .parent() anchor to add ornaments.
What makes me unsure is that other methods for drawing UI in RealityView don’t seem to work with ornaments. For example, if I add an attachment to show a view with the ornament–even when I use the .parent() anchor–the ornament is anchor to the volume, not the attachment view.
So what do we think? Is this a rendering bug? Are ornaments intended to work with attachments and presentations?
My app is getting video from UVC device, and I wish to display it in an Immersive Space. But when I open Immersive Space, the UVC capture will just stop.AI said it's due to confliction in Camera pipeline. But I don't really understand, I don't need to use any on device camera, why it conflict with my UVC...
I want to open the control center in Vision Pro’s Xcode simulator. Can I open it? If I can, please tell me how to do it. Thank you.
I am using HelloPhotogrammetry in Xcode
I can make one model with something like HelloPhotogrammetry.main([path_to_folder_of images, path_to_output/model.usdz, "-d", "medium", "-o", "unordered", "-f", "high" ])
But how would I request several models simultaneously? I only want to vary the detail.
[ ("/Users/you/Desktop/model_medium.usdz", detail: .medium), ("/Users/you/Desktop/model_full.usdz", detail: .full), ("/Users/you/Desktop/model_raw.usdz", detail: .raw ]
If I long press on an element, the sidebar disappears and then a Done appears on the screen, but nothing else changes, so what are the Environments in Vision Pro's Simulator?
I recently added pushWindow to my app, and I discovered that in visionOS 26.2 RC (23N301), pushWindow followed by dismissWindow no longer works as expected.
Specifically, if the user moves the pushed window, then when the pushed window is later dismissed, the parent window's position isn't aligned with the pushed window's new position. Its original position is restored instead.
Curiously, the bug only happens when an app is launched from the visionOS home view, and not when an app is launched from Xcode. It also doesn't happen in the visionOS 26.2 simulator.
Another interesting detail is that while the parent window is hidden, if the user long-presses the Digital Crown and then dismisses the pushed window, the parent window's position seems to be immune from the Digital Crown scene reorientation. It's restored to its original real world position.
Demo video: https://youtu.be/zR3t2ON3Wz0
I've submitted feedback as FB21287011 with a sample app and detailed repro steps.
Has anyone else encountered this issue already and figured out a workaround? It would be nice if I could get pushWindow to work correctly in my app.
Thanks everybody! 😀
Hi,
I'm trying to correct the lens distortion in frames provided by Enterprise API camera frame provider. The frames provided seem to have only in/extrinsics info, but not the distortion lookup table.
Is there some magic setting, or function to do that (I can't seem to find anything like this)? Or is there a way to use AVCameraCalibrationData together with provider?
With Xcode 26, loading ressources with RealityKit is extremely slow.
Here my project takes almost 50 seconds to load.
I also get multiple Hang detected messages in the console:
When I uncheck "Debug executable" in the schema, the same project loads in 2 seconds.
I'm using RealityKit asynchronous loading:
private static func loadFromRealityComposerPro(
named entityName: String,
fromSceneNamed sceneName: String
) async -> Entity? {
var entity: Entity?
do {
let scene = try await Entity(
named: sceneName,
in: visionPetsContentBundle
)
entity = scene.findEntity(named: entityName)
} catch {
print(
"Error loading \(entityName) from scene \(sceneName): \(error.localizedDescription)"
)
}
return entity
}
Anyone having the same problem?
Topic:
Spatial Computing
SubTopic:
General
Using Xcode v26 Beta 6 on macOS v26 Beta 25a5349a
When pressing on the home button of the visionOS simulator, I am not positioned in the middle of the room like would normally be. This occurred when moving a lot in the space to find an element added to an ImmersiveSpace.
How to resolve: restart simulator device.
See attached the pictures of the visionOSSimulatorCorrectHomePosition and the visionOSSimulatorMisallignedHomePosition.