HarmonyOS鸿蒙Next中XComponentSurface在NDK层使用后就无法进行摄像头预览
HarmonyOS鸿蒙Next中XComponentSurface在NDK层使用后就无法进行摄像头预览 正常情况：
创建一个XComponentSurface，打开摄像头预览，预览画面一切正常。
错误情况：
创建一个XComponentSurface，先在NDK层对XComponentSurface调用NativeWindowRequestBuffer函数后，再打开摄像头预览，预览画面没有出现，Hilog报错：
{OnError():47} CameraDeviceServiceCallback::OnError() is called!, errorType: 11, errorMsg: 0
请问是哪里有问题？
demo下载地址：https://gitee.com/chen_yi_ze/test-harmony-camera.git
设备：Nova 12 pro，系统：6.1.0，
更多关于HarmonyOS鸿蒙Next中XComponentSurface在NDK层使用后就无法进行摄像头预览的实战教程也可以访问 https://www.itying.com/category-93-b0.html
尊敬的开发者，您好，
在预览场景下，XComponent的Surface是图像数据缓冲区的抽象概念，相机将预览流数据写入Surface，XComponent从Surface读取数据并显示。在问题所述场景中，XComponent消费Surface缓冲区数据之前，缓冲区内的数据已经被其它接口消费掉了，导致XComponent消费失败，无法渲染预览画面。原理可以参考：实现原理
其次，不同的Surface需要设置成不同的SurfaceID，若XComponent和ImageReceiver重用了一个重用了Surface，便会产生影响。
更多关于HarmonyOS鸿蒙Next中XComponentSurface在NDK层使用后就无法进行摄像头预览的实战系列教程也可以访问 https://www.itying.com/category-93-b0.html
我觉得你说的不对。
1、ImageReceiver是用的自己内部的Surface，和UI界面上的XComponent完全没有关系，我也没有把双路预览重用一个Surface。
2、同一个Surface，如果先给相机渲染预览画面正常，用完后，再给NDK层渲染画面还是正常的。
3、同一个Surface，如果先给NDK层渲染画面正常，用完后，再给相机渲染预览画面就报错了。
请问第3种情况，我要怎么做才能正常？是需要做什么清理工作吗？还是说根本无法实现？还是说是系统BUG？
如果你不懂，请更换更高级的技术支持来解答，谢谢。
官方文档通常不会直接写“Surface 使用权冲突”这几个字，可以从 API 角色来说明：Camera 预览是把 surfaceId 作为 createPreviewOutput 的输出目标；XComponent/NativeWindow 自绘则是应用侧通过 NativeWindowRequestBuffer 取 buffer 自己写入。同一个 Surface/BufferQueue 同时给 Camera 连续输出、又被应用侧当自绘缓冲消费，容易破坏生产者/消费者关系。
如果既要显示预览又要取帧，建议按双路输出设计：XComponent Surface 只负责预览显示，ImageReceiver/类似取帧 Surface 负责算法处理。这样更接近官方 Camera 预览 + 图像接收的组合用法，也比复用一个 XComponentSurface 稳。
我取帧是从ImageReceiver这里取的，但是我surface是复用一个，但是不是同时的，我预览的时候我就只预览，预览完了，我才做别的显示，别的显示完了，我又切过来做预览。
我是用ArkGraphics在摄像头上加载3d模型的，参考一下：
@ComponentV2
struct ARWorld {
scene: scene3d.Scene | null = null;
cam: Camera | null = null;
node: Node | null | undefined = null;
@Local arContext?: arViewController.ARViewContext = undefined;
@Local bodyInfos: BodyInfo[] = [];
@Local displayWidth: number = display.getDefaultDisplaySync().width;
@Local displayHeight: number = display.getDefaultDisplaySync().height;
@Local isFrontCamera: boolean = true;
private params: arEngine.ARConfig = { type: arEngine.ARType.BODY };
private uiContext: UIContext | null = null;
private callbackImpl: ARViewCallbackImpl = new ARViewCallbackImpl();
private currentModelIndex: number = 0;
private allModels: Node[] = [];
private onBodyInfoCb: OnBodyInfoCallback = (bodyInfos: arEngine.ARBody[]) => {
if (display.getDefaultDisplaySync().width * 3 < display.getDefaultDisplaySync().height * 4) {
this.displayWidth = display.getDefaultDisplaySync().width;
this.displayHeight = this.displayWidth * DEFAULT_CONVERT_FACTOR;
} else {
this.displayHeight = display.getDefaultDisplaySync().height;
this.displayWidth = this.displayHeight * DEFAULT_CONVERT_FACTOR;
}
this.bodyInfos = bodyInfos.map((value: arEngine.ARBody) => {
let landmarks: arEngine.ARBodyLandmark2D[] = value.getLandmarks2D();
landmarks.forEach((value: arEngine.ARBodyLandmark2D) => {
value.x = value.x * this.displayWidth;
value.y = value.y * this.displayHeight;
})
let info: BodyInfo = {
trackId: value.trackId,
landmarks: landmarks
}
return info;
})
}
aboutToAppear() {
this.uiContext = this.getUIContext();
}
aboutToDisappear() {
this.stopARView();
}
private async initARView(): Promise<void> {
if (this.scene !== null) {
return;
}
const resource = $rawfile('ArkGraphics/assets/default.scene');
scene3d.Scene.load(resource).then(async (result: Scene) => {
this.scene = result;
this.cam = result.root?.getNodeByPath("Perspective Camera") as Camera;
if (this.cam) {
this.cam.position.z = 5;
this.cam.enabled = true;
}
let sceneLight = result.root?.getNodeByPath("Directional Light") as Light;
if (sceneLight) {
sceneLight.enabled = true;
sceneLight.color = { r: 1.0, g: 1.0, b: 1.0, a: 1.0 };
sceneLight.intensity = 1.5
sceneLight.rotation = {
x: 0.0,
y: 1.0,
z: 0.0,
w: 0.0
};
}
this.allModels = [];
for (const name of MODEL_LIST) {
const modelNode = result.root?.getNodeByPath(name);
if (modelNode) {
modelNode.visible = false;
this.allModels.push(modelNode);
}
}
this.node = this.allModels[this.currentModelIndex] ?? null;
if (this.node) {
this.node.visible = true;
}
this.scene.environment.backgroundType = EnvironmentBackgroundType.BACKGROUND_NONE;
let viewContext: arViewController.ARViewContext = new arViewController.ARViewContext();
viewContext.scene = result;
this.callbackImpl = new ARViewCallbackImpl();
this.callbackImpl.setCallback(this.onBodyInfoCb);
this.callbackImpl.setIsFrontCamera(this.isFrontCamera);
this.callbackImpl.setTargetNode(this.node);
viewContext.callback = this.callbackImpl;
viewContext.config = {
type: arEngine.ARType.BODY,
planeFindingMode: arEngine.ARPlaneFindingMode.DISABLED,
powerMode: arEngine.ARPowerMode.NORMAL,
semanticMode: arEngine.ARSemanticMode.NONE,
poseMode: arEngine.ARPoseMode.GRAVITY,
depthMode: arEngine.ARDepthMode.DISABLED,
meshMode: arEngine.ARMeshMode.DISABLED,
focusMode: arEngine.ARFocusMode.AUTO,
maxDetectedBodyNum: this.params?.maxDetectedBodyNum ?? 1,
cameraLensFacing: this.isFrontCamera ? 1 : 0
};
viewContext.init().then(() => {
this.arContext = viewContext;
console.info('Succeeded in initializing ARView.');
}).catch((err: BusinessError) => {
console.error(`Failed to init ARView. Code is ${err.code}, message is ${err.message}.`);
this.scene?.destroy();
this.scene = null;
this.cam = null;
this.node = null;
this.allModels = [];
this.arContext = undefined;
});
console.info(`AR scene loaded, active model: ${MODEL_LIST[this.currentModelIndex]}`);
}).catch((reason: string) => {
console.error(`init error: ${reason}`);
});
}
private toggleCamera(): void {
this.stopARView();
this.arContext = undefined;
this.isFrontCamera = !this.isFrontCamera;
this.scene = null;
this.initARView();
}
private stopARView(): void {
if (!this.arContext) {
return;
}
try {
this.arContext.destroy();
this.arContext = undefined;
if (this.scene) {
this.scene.destroy();
this.scene = null;
}
this.cam = null;
this.node = null;
this.allModels = [];
this.bodyInfos = [];
} catch (error) {
const err: BusinessError = error as BusinessError;
console.error(`Failed to stop context. Code is ${err.code}, message is ${err.message}`);
}
}
private pauseARView(): void {
if (!this.arContext) {
return;
}
try {
this.arContext.pause();
} catch (error) {
const err: BusinessError = error as BusinessError;
console.error(`Failed to pause context. Code is ${err.code}, message is ${err.message}.`);
}
}
private resumeARView(): void {
if (!this.arContext) {
return;
}
try {
this.arContext.resume();
} catch (error) {
const err: BusinessError = error as BusinessError;
console.error(`Failed to resume context. Code is ${err.code}, message is ${err.message}.`);
}
}
build() {
NavDestination() {
Stack() {
Column() {
RelativeContainer() {
if (this.arContext) {
ARView({ context: this.arContext })
.height(FULL_SCREEN_SIZE)
.width(FULL_SCREEN_SIZE)
.alignRules({
center: { anchor: '__container__', align: VerticalAlign.Center },
middle: { anchor: '__container__', align: HorizontalAlign.Center }
})
}
}
}
.width(this.uiContext ? this.uiContext.px2vp(this.displayWidth) : FULL_SCREEN_SIZE)
.height(this.uiContext ? this.uiContext.px2vp(this.displayHeight) : FULL_SCREEN_SIZE)
.justifyContent(FlexAlign.Center)
.alignItems(HorizontalAlign.Center)
if (!this.bodyInfos.length) {
Column() {
SymbolGlyph($r('sys.symbol.person_fill'))
.fontSize(52)
.renderingStrategy(SymbolRenderingStrategy.SINGLE)
.fontColor([Color.White])
.opacity(0.5)
Text('请将全身置于摄像头前')
.fontSize(16)
.fontColor('#FFFFFF')
.opacity(0.5)
.margin({ top: 16 })
}
.position({ x: '50%', y: '40%' })
.translate({ x: '-50%', y: '-50%' })
}
Button() {
Column() {
SymbolGlyph($r('sys.symbol.camera_fill'))
.renderingStrategy(SymbolRenderingStrategy.SINGLE)
.fontSize(24)
.fontColor([Color.White])
Text(this.isFrontCamera ? '前置' : '后置')
.fontSize(11)
.fontColor('#FFFFFF')
.margin({ top: 6 })
}
}
.width(68)
.height(68)
.borderRadius(34)
.backgroundColor('#66000000')
.shadow({ radius: 16, color: '#40000000', offsetX: 0, offsetY: 6 })
.position({ x: '50%', y: '92%' })
.translate({ x: '-50%', y: '-50%' })
.onClick(() => {
this.toggleCamera();
})
}
.width(FULL_SCREEN_SIZE)
.height(FULL_SCREEN_SIZE)
}
.onAppear(() => {
})
.onWillDisappear(() => {
this.stopARView();
})
.onShown(() => {
this.resumeARView();
})
.onHidden(() => {
this.pauseARView();
})
.onReady(ctx => {
if (ctx.pathInfo && ctx.pathInfo.param) {
const routeParams = ctx.pathInfo.param as Record<string, Object>;
const modelIdx = routeParams['modelIndex'] as number;
if (modelIdx !== undefined && modelIdx >= 0 && modelIdx < MODEL_LIST.length) {
this.currentModelIndex = modelIdx;
}
}
this.params.type = arEngine.ARType.BODY;
this.initARView();
})
.hideTitleBar(true)
.hideBackButton(false)
.hideToolBar(true)
}
@Builder
drawBodyPerception() {
Shape() {
ForEach(this.bodyInfos, (bodyInfo: BodyInfo, idx: number) => {
this.drawBodyBones(arLandmarksToMap(bodyInfo.landmarks));
this.drawBodyLandmarks(bodyInfo.landmarks);
})
}
.width(this.uiContext ? this.uiContext.px2vp(this.displayWidth) : FULL_SCREEN_SIZE)
.height(this.uiContext ? this.uiContext.px2vp(this.displayHeight) : FULL_SCREEN_SIZE)
}
@Builder
drawBodyLandmarks(bodyLandmarks: arEngine.ARBodyLandmark2D[]) {
ForEach(bodyLandmarks, (landmark: arEngine.ARBodyLandmark2D, index: number) => {
Circle({ width: 4, height: 4 })
.position({
x: this.uiContext ? this.uiContext.px2vp(landmark.x) : 0,
y: this.uiContext ? this.uiContext.px2vp(landmark.y) : 0
})
.fillOpacity(1)
.fill(Color.White)
})
}
@Builder
drawBodyBones(bodyLandmarks: Map<arEngine.ARBodyLandmarkType, arEngine.ARBodyLandmark2D>) {
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.NOSE, arEngine.ARBodyLandmarkType.LEFT_SHOULDER,
Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.NOSE, arEngine.ARBodyLandmarkType.RIGHT_SHOULDER,
Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_SHOULDER,
arEngine.ARBodyLandmarkType.RIGHT_SHOULDER, Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_SHOULDER,
arEngine.ARBodyLandmarkType.RIGHT_HIP, Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_HIP, arEngine.ARBodyLandmarkType.LEFT_HIP,
Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_HIP,
arEngine.ARBodyLandmarkType.LEFT_SHOULDER, Color.Orange);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_SHOULDER,
arEngine.ARBodyLandmarkType.LEFT_ELBOW, Color.Green);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_ELBOW, arEngine.ARBodyLandmarkType.LEFT_WRIST,
Color.Green);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_HIP, arEngine.ARBodyLandmarkType.LEFT_KNEE,
Color.Green);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.LEFT_KNEE, arEngine.ARBodyLandmarkType.LEFT_ANKLE,
Color.Green);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_SHOULDER,
arEngine.ARBodyLandmarkType.RIGHT_ELBOW, Color.Blue);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_ELBOW,
arEngine.ARBodyLandmarkType.RIGHT_WRIST, Color.Blue);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_HIP, arEngine.ARBodyLandmarkType.RIGHT_KNEE,
Color.Blue);
this.drawBodyBoneLine(bodyLandmarks, arEngine.ARBodyLandmarkType.RIGHT_KNEE,
arEngine.ARBodyLandmarkType.RIGHT_ANKLE, Color.Blue);
}
@Builder
drawBodyBoneLine(bodyLandmarks: Map<arEngine.ARBodyLandmarkType, arEngine.ARBodyLandmark2D>,
start: arEngine.ARBodyLandmarkType, end: arEngine.ARBodyLandmarkType, color: Color) {
if (bodyLandmarks.has(start) && bodyLandmarks.has(end) && this.uiContext) {
Line()
.startPoint([this.uiContext.px2vp(bodyLandmarks.get(start)?.x),
this.uiContext.px2vp(bodyLandmarks.get(start)?.y)])
.endPoint([this.uiContext.px2vp(bodyLandmarks.get(end)?.x), this.uiContext.px2vp(bodyLandmarks.get(end)?.y)])
.stroke(color)
.strokeWidth(3)
}
}
}
即使只在预览前调用一次 NativeWindowRequestBuffer，也可能已经把这一路 Surface 先放进应用侧 dequeue/queue 的使用链路了；相机预览输出同样要管理这个 Surface 对应的 BufferQueue，两边混用就容易出现 session 建好了但没有画面的情况。
建议不要让同一个 XComponentSurface 同时承担 NDK 自绘取 buffer 和 Camera 预览输出。预览用 XComponentSurface；如果还要拿预览帧，按双路预览思路再单独创建 ImageReceiver Surface 作为第二路输出。若业务必须自绘同一个 Surface，也要等相机会话 stop/release 之后再做 NativeWindow 操作。可以先做一个最小验证：完全移除 NativeWindowRequestBuffer 后预览恢复，就基本能确认是 Surface 使用权冲突。
问题大概率在 Surface 使用权上。XComponentSurface 交给 Camera 做预览输出后，就不要再先用 NativeWindowRequestBuffer 把同一个 Surface 当成 NDK 自绘窗口消费了；相机预览也需要向这个 Surface 的 BufferQueue 送帧，两边同时操作很容易导致预览输出创建成功但实际无画面。建议拆成两路：XComponent 的 surfaceId 只给 createPreviewOutput 做预览显示；如果你还要在 NDK 层拿帧/处理图像，用 ImageReceiver 或第二路预览 Surface，而不是提前 request 同一个 XComponent buffer。
HarmonyOS Next 中，XComponentSurface 在 NDK 层被获取并用于渲染后，其内部 Surface 对象状态会从空闲变为占用状态。摄像头预览需要通过 CameraKit 绑定同一 Surface 才能输出帧数据，但此时 Surface 已被 NDK 层持有并可能修改了缓冲区配置或显示模式，导致 CameraKit 无法再正确识别或绑定该 Surface，从而预览失败。这是资源独占与生命周期管理的约束所致。
在 NDK 层直接对 XComponent 的 Surface 调用 NativeWindowRequestBuffer 后，该 Surface 的 buffer 被锁定占用，导致摄像头服务无法再从该 Surface 获取可用的图形缓冲区，因此预览初始化失败，并触发 errorType: 11（通常为 Buffer 相关错误）。
XComponent 的 Surface 由框架管理，除非在自渲染场景下严格按照 NAPI 规范进行 buffer 的申请、绘制与释放，否则不应主动调用 buffer 请求接口，否则会破坏 Surface 的可用状态，使摄像头输出目标失效。
