A permission based media system to perform operations that include: presenting a first media object at a client device associated with a user account, the first media object including a reference that identifies the user account; receiving an input that selects the first media object from the client device; determining a permission of the user account based on the reference that identifies the user account; presenting a set of options based on the permission associated with the user account; receiving a selection of an option from among the set of options; and generating a second media object based on the first media object and the selection of the option, according to certain embodiments.
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
2.
IMAGE GENERATION USING SURFACE-BASED NEURAL SYNTHESIS
Aspects of the present disclosure involve a system and a method for performing operations comprising: receiving a two-dimensional continuous surface representation of a three-dimensional object, the continuous surface comprising a plurality of landmark locations; determining a first set of soft membership functions based on a relative location of points in the two-dimensional continuous surface representation and the landmark locations; receiving a two-dimensional input image, the input image comprising an image of the object; extracting a plurality of features from the input image using a feature recognition model; generating an encoded feature representation of the extracted features using the first set of soft membership functions; generating a dense feature representation of the extracted features from the encoded representation using a second set of soft membership functions; and processing the second set of soft membership functions and dense feature representation using a neural image decoder model to generate an output image.
G06T 5/50 - Amélioration ou restauration d'image utilisant plusieurs images, p. ex. moyenne ou soustraction
G06V 10/74 - Appariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques
G06V 10/80 - Fusion, c.-à-d. combinaison des données de diverses sources au niveau du capteur, du prétraitement, de l’extraction des caractéristiques ou de la classification
Methods and systems are disclosed for using generative machine learning models to generate fashion items for avatars. The methods and systems present a graphical user interface (GUI) comprising icons representing different types of avatar fashion items. The methods and systems receive input that selects an individual icon corresponding to an individual avatar fashion item. The methods and systems, in response to receiving the input, receive a selection of an individual texture from a texture selection region, the texture selection region comprising a first set of predefined textures and an option associated with a second set of textures generated based on a prompt by a generative machine learning model. The methods and systems apply the individual avatar fashion item with the individual texture to an avatar depicted in the GUI.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
A63F 13/53 - Commande des signaux de sortie en fonction de la progression du jeu incluant des informations visuelles supplémentaires fournies à la scène de jeu, p. ex. en surimpression pour simuler un affichage tête haute [HUD] ou pour afficher une visée laser dans un jeu de tir
A63F 13/795 - Aspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour trouver d’autres joueursAspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour constituer une équipeAspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu pour fournir une "liste d’amis"
A63F 13/87 - Communiquer avec d’autres joueurs, p. ex. par courrier électronique ou messagerie instantanée
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
A method of dissipating heat generated by imaging devices and processing devices of a wearable electronic eyewear device includes providing a first heat sink thermally connecting the imaging devices to a frame of the eyewear device to sink heat to the frame and providing a second heat sink thermally connecting the processing devices to respective temples of the eyewear device to sink heat to the respective temples. The first and second heat sinks are thermally insulated from each other to direct the heat to different portions of the eyewear device. The processing devices may include a first co-processor disposed in a first temple connected to a first end of the frame and a second co-processor disposed in a second temple connected to a second end of the frame. The resulting eyewear device spreads the heat from heat generating devices over a larger area to minimize overall heating.
A personalized preview system to receive a request to access a collection of media items from a user of a user device. Responsive to receiving the request to access the collection of media items, the personalized preview system accesses user profile data associated with the user, wherein the user profile data includes an image. For example, the image may comprise a depiction of a face, wherein the face comprises a set of facial landmarks. Based on the image, the personalized preview system generates one or more media previews based on corresponding media templates and the image, and displays the one or more media previews within a presentation of the collection of media items at a client device of the user.
Systems and methods are provided for generating an augmented reality (AR) experience. The systems and methods receive a video depicting movement of a humanoid and a target image depicting an object. The systems and methods process, by a generative machine learning (ML) model, the video and the target image to generate a new video depicting the object performing the movement. The systems and methods generate the AR experience using the new video to overlay a face of a user on a portion of the new video.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing audio with captured video clips. The program and method provide for displaying, by a messaging application, a capture user interface for capturing video; providing a camera mode selection element which is selectable to switch between a first camera mode for capturing a single video clip and a second camera mode for capturing multiple video clips, to generate a media content item; providing an audio selection element which is selectable to select an audio track for the media content item; receiving, via the camera mode selection element, first user input selecting the second camera mode; receiving, via the audio selection element, second user input selecting the audio track; and providing for capturing multiple video clips in association with the selected audio track for generating the media content item.
A head-wearable extended reality (XR) device includes an optical assembly. The optical assembly has a display and an optical element. The display is provided to display virtual content to a user of the XR device. The optical element is provided to direct the virtual content from the display along an optical path to an eye of the user. The optical element includes a first portion and a second portion. The first portion provides a first focus distance that corresponds to a first viewing zone of the display. The second portion provides a second focus distance that differs from the first focus distance and corresponds to a second viewing zone of the display.
An occlusion detection system to perform operations that include: capturing image data that depicts an environment at a client device, the environment including a target object at a position within the environment; causing display of a presentation of the environment at the client device, the presentation of the environment including a display of the target object at the position within the environment; detecting a first attribute of the display of the target object at the client device; performing a comparison of the first attribute of the display of the target object and a second attribute associated with the target object; and detecting an occlusion based on the comparison.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06T 7/90 - Détermination de caractéristiques de couleur
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 10/40 - Extraction de caractéristiques d’images ou de vidéos
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
H04L 51/222 - Surveillance ou traitement des messages en utilisant des informations de localisation géographique, p. ex. des messages transmis ou reçus à proximité d'un certain lieu ou d'une certaine zone
H04W 4/021 - Services concernant des domaines particuliers, p. ex. services de points d’intérêt, services sur place ou géorepères
H04W 4/30 - Services spécialement adaptés à des environnements, à des situations ou à des fins spécifiques
The media preview system receives media content from one or more client devices, generates a preview of the media content, associates a coded image with the preview within a database associated with the media preview system, detects scans of the coded image from client devices, and causes display of the preview at the client devices in response to detecting the scan.
G06F 16/70 - Recherche d’informationsStructures de bases de données à cet effetStructures de systèmes de fichiers à cet effet de données vidéo
G06Q 30/0207 - Remises ou incitations, p. ex. coupons ou rabais
H04L 65/612 - Diffusion en flux de paquets multimédias pour la prise en charge des services de diffusion par flux unidirectionnel, p. ex. radio sur Internet pour monodiffusion [unicast]
An antenna that is coupled to and integrated with a projector, such as a projector included with smart glasses including eyewear. The projector has a housing and includes optical components configured to display an image. At least one antenna is coupled to the projector, wherein the optical components operate and function as an antenna substrate. The optical components are nonmetallic such that the antenna generates a strong E-field. The antenna may be coupled to the projector housing, such as on the inside or the outside surface of the housing. Multiple antennas can be included to generate multiple resonances simultaneously in different frequency bands.
Disclosed are systems, methods, and non-transitory computer-readable media for continuous surface and depth estimation. A continuous surface and depth estimation system determines the depth and surface normal of physical objects by using stereo vision limited within a predetermined window.
Embodiments described herein include an expressive icon system to present an animated graphical icon, wherein the animated graphical icon is generated by capture facial tracking data at a client device. In some embodiments, the system may track and capture facial tracking data of a user via a camera associated with a client device (e.g., a front facing camera, or a paired camera), and process the facial tracking data to animate a graphical icon.
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
H04M 1/72427 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité pour donner accès à des jeux ou à des animations graphiques
H04M 1/7243 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité avec des moyens interactifs de gestion interne des messages
H04M 1/72469 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles pour faire fonctionner le dispositif en sélectionnant des fonctions à partir de plusieurs éléments affichés, p. ex. des menus ou des icônes
A multimodal video generation framework (MMVID) that benefits from text and images provided jointly or separately as input. Quantized representations of videos are utilized with a bidirectional transformer with multiple modalities as inputs to predict a discrete video representation. A new video token trained with self-learning and an improved mask-prediction algorithm for sampling video tokens is used to improve video quality and consistency. Text augmentation is utilized to improve the robustness of the textual representation and diversity of generated videos. The framework incorporates various visual modalities, such as segmentation masks, drawings, and partially occluded images. In addition, the MMVID extracts visual information as suggested by a textual prompt.
H04N 21/472 - Interface pour utilisateurs finaux pour la requête de contenu, de données additionnelles ou de servicesInterface pour utilisateurs finaux pour l'interaction avec le contenu, p. ex. pour la réservation de contenu ou la mise en place de rappels, pour la requête de notification d'événement ou pour la transformation de contenus affichés
A system to provide users with a means for accessing media content directly, by performing operations that include: causing display of a media item within a graphical user interface at a client device, the graphical user interface including a set of graphical elements; receiving a selection of a graphical element from among the set of graphical elements within the graphical user interface; generating a reference to the media item based on the selection of the graphical element; encoding a matrix barcode with the reference to the media item; and generating a presentation of the media item that includes a display of the matrix barcode at a position within the media item.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/04847 - Techniques d’interaction pour la commande des valeurs des paramètres, p. ex. interaction avec des règles ou des cadrans
G06F 9/451 - Dispositions d’exécution pour interfaces utilisateur
G06F 16/955 - Recherche dans le Web utilisant des identifiants d’information, p. ex. des localisateurs uniformisés de ressources [uniform resource locators - URL]
16.
ADJUSTABLE DISPLAY ARRANGEMENT FOR EXTENDED REALITY DEVICES
A head-wearable extended reality (XR) device includes a display arrangement. The display arrangement has a display to display virtual content, and also has one or more optical elements to direct the virtual content along an optical path to an eye of a user of the XR device. The virtual content is presented in a virtual content field of view. The display arrangement further includes an adjustment mechanism to alter the optical path so as to adjust the virtual content field of view between at least two display modes.
A button-switch assembly provides a preloaded force design with an enhanced tactile feel while also providing a non-wobbly (stabilized) configuration and water/dust protection functions. Features of the button-switch assembly include excellent tactile feel through a stack up of a soft rubber layer of a deflection web and a hard PET film shim layer, a consistent pre-loaded push force through use of an angled deflection web, a button flange that minimizes rotation of the button while providing a consistent tactile feel even when the edge of the button is depressed, double sided sealing adhesive layers that seal off the opening in the housing for accepting the button to prevent water/dust from entering the opening, and gluing the button to the rubber deflection web in variable thicknesses to provide a stable tension force to minimize wobble of the button when depressed.
H01H 13/14 - Organes d'actionnement, p. ex. bouton-poussoir
H01H 13/06 - Enveloppes étanches à la poussière, aux projections, aux éclaboussures, à l'eau ou aux flammes
H01H 13/705 - Interrupteurs ayant un organe moteur à mouvement rectiligne ou des organes adaptés pour pousser ou tirer dans une seule direction, p. ex. interrupteur à bouton-poussoir ayant une pluralité d'éléments moteurs associés à différents jeux de contacts, p. ex. claviers avec des contacts portés par ou formés à partir de couches dans une structure multicouche, p. ex. interrupteurs à membrane caractérisés par la structure, le montage ou l'agencement des organes d'actionnement, p. ex. des boutons-poussoirs ou des touches
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: accessing, by a first application implemented on a client device, data collected from one or more entropy sources; causing a second application implemented on the client device to access the data collected from the one or more entropy sources; generating a shared cryptographic key using the data collected from one or more entropy sources; establishing a communication channel between the first application and the second application; and exchanging, over the communication channel between the first application and the second application, one or more messages that have been encrypted using the shared cryptographic key.
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Peripherals; Augmented reality glasses; Augmented reality headsets; Computer hardware with embedded operating system software; Computer hardware, peripherals and software for remotely accessing, capturing, transmitting and displaying pictures, video, audio and data; Downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; Downloadable software for setting up, configuring, and controlling wearable computer hardware and peripheral devices in the field of augmented reality; Downloadable computer operating software for augmented reality; Downloadable mobile operating system software; Downloadable computer operating system software; Downloadable computer operating system for operating augmented reality devices; Downloadable computer software offering backend components for augmented reality software development including data storage and database integration, user authentication services, real-time capabilities, application programming interfaces (APIs), vector embeddings, and backward compatibility Providing temporary use of online non-downloadable middleware for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between computer peripheral devices and operating systems; Platform as a service (PAAS) featuring computer software offering backend components for software developers; Backend as a service (BAAS) services featuring a software platform that offers backend components, including data storage and database integration, user authentication services, real-time capabilities, application programming interfaces (APIs), vector embeddings, and backward compatibility
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Computer peripherals; Augmented reality glasses; Augmented reality headsets; Computer hardware, peripherals and recorded software for remotely accessing, capturing, transmitting and displaying pictures, video, audio and data; Software for setting up, configuring, and controlling wearable computer hardware and peripherals; Software for setting up, configuring, and controlling wearable computer hardware and peripheral devices in the field of augmented reality; Downloadable computer operating software for augmented reality; Downloadable mobile operating system software; Downloadable computer operating system software; Downloadable computer operating system for operating augmented reality devices; Downloadable communications software for connecting computer network users; Downloadable computer software for organizing and viewing digital images and photographs; Downloadable computer operating system software for virtual environments; Downloadable operating system programs; Recorded operating system programs; Recorded computer operating system software; Computer hardware with embedded operating system software; Downloadable computer software for use as an application programming interface (API); Downloadable computer software for creating digital animation and special effects of images; Downloadable communication software for providing access to the Internet; Downloadable computer search engine software; Downloadable software for browsing the internet and accessing search engines; Downloadable computer software for geographic mapping, location mapping, spatial mapping, and spatial computing Providing temporary use of online non-downloadable middleware for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between computer peripheral devices and operating systems; Non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; Non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripheral devices in the field of augmented reality; Non-downloadable computer operating software for augmented reality; Non-downloadable mobile operating system software; Non-downloadable computer operating system software; Non-downloadable computer operating system for operating augmented reality devices; non-downloadable communications software for connecting computer network users; Non-downloadable computer software for organizing and viewing digital images and photographs; Non-downloadable computer operating system software for virtual environments; Non-downloadable operating system programs; Non-downloadable computer software for use as an application programming interface (API); Non-downloadable computer software for creating digital animation and special effects of images; Non-downloadable communication software for providing access to the Internet; Non-downloadable computer search engine software; Providing temporary use of non-downloadable cloud-based software for connecting, operating, and managing networked wearable computer peripherals in the internet of things (IoT); Design, maintenance, development and updating of computer software; Development, maintenance and updating of a telecommunication network search engine; Software as a service (SAAS) services featuring software for browsing the internet and accessing search engines; Provision of Internet search engines; Providing a website featuring a search engine for accessing online content and content sharing; Providing temporary use of on-line non-downloadable software for geographic mapping, location mapping, spatial mapping, and spatial computing
(1) Computer peripherals; Augmented reality glasses; Augmented reality headsets; Computer hardware, peripherals and recorded software for remotely accessing, capturing, transmitting and displaying pictures, video, audio and data; Software for setting up, configuring, and controlling wearable computer hardware and peripherals; Software for setting up, configuring, and controlling wearable computer hardware and peripheral devices in the field of augmented reality; Downloadable computer operating software for augmented reality; Downloadable mobile operating system software; Downloadable computer operating system software; Downloadable computer operating system for operating augmented reality devices; Downloadable communications software for connecting computer network users; Downloadable computer software for organizing and viewing digital images and photographs; Downloadable computer operating system software for virtual environments; Downloadable operating system programs; Recorded operating system programs; Recorded computer operating system software; Computer hardware with embedded operating system software; Downloadable computer software for use as an application programming interface (API); Downloadable computer software for creating digital animation and special effects of images; Downloadable communication software for providing access to the Internet; Downloadable computer search engine software; Downloadable software for browsing the internet and accessing search engines; Downloadable computer software for geographic mapping, location mapping, spatial mapping, and spatial computing (1) Providing temporary use of online non-downloadable middleware for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between augmented reality devices and operating systems; Providing temporary use of online non-downloadable software for providing an interface between computer peripheral devices and operating systems; Non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripherals; Non-downloadable software for setting up, configuring, and controlling wearable computer hardware and peripheral devices in the field of augmented reality; Non-downloadable computer operating software for augmented reality; Non-downloadable mobile operating system software; Non-downloadable computer operating system software; Non-downloadable computer operating system for operating augmented reality devices; non-downloadable communications software for connecting computer network users; Non-downloadable computer software for organizing and viewing digital images and photographs; Non-downloadable computer operating system software for virtual environments; Non-downloadable operating system programs; Non-downloadable computer software for use as an application programming interface (API); Non-downloadable computer software for creating digital animation and special effects of images; Non-downloadable communication software for providing access to the Internet; Non-downloadable computer search engine software; Providing temporary use of non-downloadable cloud-based software for connecting, operating, and managing networked wearable computer peripherals in the internet of things (IoT); Design, maintenance, development and updating of computer software; Development, maintenance and updating of a telecommunication network search engine; Software as a service (SAAS) services featuring software for browsing the internet and accessing search engines; Provision of Internet search engines; Providing a website featuring a search engine for accessing online content and content sharing; Providing temporary use of on-line non-downloadable software for geographic mapping, location mapping, spatial mapping, and spatial computing
09 - Appareils et instruments scientifiques et électriques
42 - Services scientifiques, technologiques et industriels, recherche et conception
Produits et services
Downloadable software; downloadable mobile applications;
computer programs and downloadable computer software using
artificial intelligence for natural language processing,
generation, understanding and analysis; downloadable
computer programs and downloadable computer software for
machine learning; downloadable computer programs and
downloadable computer software for image recognition and
generation; downloadable computer programs and downloadable
computer software using artificial intelligence for music
generation and suggestions; downloadable computer programs
and downloadable computer software for artificial
intelligence, namely, computer software for developing,
running and analyzing algorithms that are able to learn to
analyze, classify, and take actions in response to exposure
to data; downloadable computer software for machine-learning
based language and speech processing; downloadable software
using artificial intelligence for image recognition and
generation; downloadable software using artificial
intelligence for text recognition and generation;
downloadable computer software using artificial intelligence
for image and video generation, editing and retouching;
downloadable computer software using artificial intelligence
(AI) for the generation of text, images, photos, videos,
audio, and multimedia content; downloadable computer
software using artificial intelligence (AI) for connecting
consumers with targeted promotional advertisements;
downloadable computer software using artificial intelligence
(AI) for the generation of advertisements and promotional
materials; downloadable computer software using artificial
intelligence (AI) for creating and generating text. Providing online non-downloadable software; research and
development services; research and development services in
the field of artificial intelligence; providing on-line
non-downloadable software using artificial intelligence (AI)
for natural language processing, generation, understanding,
and analysis; providing on-line non-downloadable software
for machine learning; providing on-line non-downloadable
software for image recognition and generation; providing
on-line non-downloadable software for developing, running
and analyzing algorithms that are able to learn to analyze,
classify, and take actions in response to exposure to data;
software as a service (SaaS) services featuring software for
using language models; providing on-line non-downloadable
software for machine-learning based language and speech
processing; providing on-line non-downloadable software
using artificial intelligence (AI) for image recognition and
generation; providing on-line non-downloadable software
using artificial intelligence (AI) for text recognition and
generation; providing on-line non-downloadable software for
the generation of advertisements and promotional materials;
providing on-line non-downloadable software using artificial
intelligence (AI) for music generation and suggestions;
providing on-line non-downloadable software using artificial
intelligence (AI) for image and video generation, editing
and retouching; providing on-line non-downloadable software
using artificial intelligence (AI) for the generation of
text, images, photos, videos, audio, and multimedia content;
providing on-line non-downloadable software using artificial
intelligence (AI) for connecting consumers with promotional
advertisements; providing on-line non-downloadable software
using artificial intelligence (AI) for the generation of
advertisements and promotional materials; providing on-line
non-downloadable software using artificial intelligence (AI)
for creating and generating text.
23.
SINGLE IMAGE THREE-DIMENSIONAL HAIR RECONSTRUCTION
A system to enable 3D hair reconstruction and rendering from a single reference image which performs a multi-stage process that utilizes both a 3D implicit representation and a 2D parametric embedding space.
A candidate content item is identified for integration into a content collection. The candidate content item is associated with a first value. Using at least one machine learning model, a select value and a skip value are automatically generated for the candidate content item. The select value indicates a likelihood that the user will select the candidate content item, and the skip value indicates a likelihood that the user will bypass the candidate content item. A second value is generated for the candidate content item based on the first value, the select value, and the skip value. The candidate content item is automatically selected from a plurality of candidate content items based on the second value meeting at least one predetermined criterion. The selected candidate content item is then automatically integrated into the content collection, which is caused to be presented on a device of a user.
Eyewear devices that include two SoCs that share processing workload. Instead of using a single SoC located either on the left or right side of the eyewear devices, the two SoCs have different assigned responsibilities to operate different devices and perform different processes to balance workload. In one example, the eyewear device utilizes a first SoC to operate displays, and it performs three-dimensional graphics and compositing. A second SoC operates first and second color cameras, first and second computer vision (CV) cameras, an operating system (OS), CV algorithms, and visual odometry (VIO), and it performs hand gesture tracking of the user and provides depth from stereo images. This configuration provides organized logistics to efficiently operate various features, and balanced power consumption.
A head-wearable extended reality (XR) device includes an optical assembly. The optical assembly has a display and an optical element. The display is provided to display virtual content to a user of the XR device. The optical element is provided to direct the virtual content from the display along an optical path to an eye of the user. The optical element includes a first portion and a second portion. The first portion provides a first focus distance that corresponds to a first viewing zone of the display. The second portion provides a second focus distance that differs from the first focus distance and corresponds to a second viewing zone of the display.
A head-wearable extended reality (XR) device includes a display arrangement. The display arrangement has a display to display virtual content, and also has one or more optical elements to direct the virtual content along an optical path to an eye of a user of the XR device. The virtual content is presented in a virtual content field of view. The display arrangement further includes an adjustment mechanism to alter the optical path so as to adjust the virtual content field of view between at least two display modes.
H04N 13/344 - Affichage pour le visionnement à l’aide de lunettes spéciales ou de visiocasques avec des visiocasques portant des affichages gauche et droit
H04N 13/361 - Reproduction d’images stéréoscopiques mixtesReproduction d’images stéréoscopiques et monoscopiques mixtes, p. ex. une fenêtre avec une image stéréoscopique en superposition sur un arrière-plan avec une image monoscopique
H04N 13/383 - Suivi des spectateurs pour le suivi du regard, c.-à-d. avec détection de l’axe de vision des yeux du spectateur
A three-dimensional (3D) scene is generated from non-aligned generic camera priors by producing a tri-plane representation for an input scene received in random latent code, obtaining a camera posterior including posterior parameters representing color and density data from the random latent code and from generic camera priors without alignment assumptions, and volumetrically rendering an image of the input scene from the color and density data to provide a scene having pixel colors and depth values from an arbitrary camera viewpoint. A depth adaptor processes depth values to generate an adapted depth map that bridges domains of rendered and estimated depth maps for the image of the input scene. The adapted depth map, color data, and scene geometry information from an external dataset are provided to a discriminator for selection of a 3D representation of the input scene.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06T 7/50 - Récupération de la profondeur ou de la forme
G06T 7/90 - Détermination de caractéristiques de couleur
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
29.
SHOOTING INTERACTION USING AUGMENTED REALITY CONTENT IN A MESSAGING SYSTEM
The subject technology receives a set of frames. The subject technology detect a first gesture correspond to an open trigger finger gesture. The subject technology receives a second set of frames. The subject technology detects from the second set of frames, a second gesture correspond to a closed trigger finger gesture. The subject technology detects a location and a position of a representation of a finger from the closed trigger finger gesture. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders a movement of the first virtual object along a vector away from the location and the position of the representation of the finger within a first scene. The subject technology provides for display the rendered movement of the first virtual object along the vector within the first scene.
Various embodiments provide systems, methods, devices, and instructions for protected data use in a third-party software application, where use can be enabled while maintaining protection of the protected data from the third party software application. In particular, various embodiments provide a software application architecture that permits a data party that owns or maintains protected data to support a software development ecosystem where a third-party can develop a third-party software application that uses the protected data while denying the third-party access to the protected data.
G06F 21/62 - Protection de l’accès à des données via une plate-forme, p. ex. par clés ou règles de contrôle de l’accès
G06F 21/53 - Contrôle des utilisateurs, des programmes ou des dispositifs de préservation de l’intégrité des plates-formes, p. ex. des processeurs, des micrologiciels ou des systèmes d’exploitation au stade de l’exécution du programme, p. ex. intégrité de la pile, débordement de tampon ou prévention d'effacement involontaire de données par exécution dans un environnement restreint, p. ex. "boîte à sable" ou machine virtuelle sécurisée
A support arm assembly for a head-worn device provides radio frequency (RF) shielding for a projector. A metal support arm, configured to structurally attach to a rear structural element and an optical element holder of the head-worn device, forms a rear face, a bottom face, and a top face of an enclosure. A metal front face of the enclosure attaches to the optical element holder, and defines a front aperture for permitting passage of light from an exit pupil of the projector toward an input optical element. The metal support arm forms a structural support joining the optical element holder to the rear structural element without placing mechanical load on the projector. A first side face of the enclosure and a second side face of the enclosure are electrically coupled to the metal support arm.
The present invention relates to improvements to systems and methods for determining a current location of a client device, and for identifying and selecting appropriate geo-fences based on the current location of the client device. An improved geo-fence selection system performs operations that include associating media content with a geo-fence that encompasses a portion of a geographic region, sampling location data from a client device, defining a boundary based on the sampled location data from the client device, detecting an overlap between the boundary and the geo-fence, retrieving the media content associated with the geo-fence, and loading the media content at a memory location of the client device, in response to detecting the overlap.
Systems, methods, and computer readable media for 3D content display using head-wearable apparatuses. Example methods include a head-wearable apparatus that is configured to determine a position for a content item on a closest curved line, of a plurality of curved lines, to the head-wearable apparatus that has space for the content item. The method includes adjusting a shape of the content item based on the position of the content item on the closest curved line and a user view of a user of the head-wearable apparatus. The method includes causing the adjusted content item to be displayed on a display of the head-wearable apparatus at the position on the closest curved line. The curved lines are either higher or lower as the curved lines goes away from the head-wearable apparatus. Additionally, the curved line or the content item may be adjusted with a random movement for an organic appearance.
Image augmentation effects are provided on a device that includes a display and a camera. A simplified augmented reality effect is applied to a stream of images captured by the camera, to generate a preview stream of images. The preview stream of images is displayed on the display. A second stream of images corresponding to the first stream of images is saved to an initial video file. A full augmented reality effect, corresponding to the simplified augmented reality affect, is then applied to the second stream of images to generate a fully-augmented stream of images, which are saved to a further video file. The further video file can then be played back on the display to show the final, fully augmented reality effect as applied to the stream of images.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving a request to present augmented reality content in association with an image captured by a device camera, the image depicting a user's face; accessing an augmented reality content item applying makeup to the face, the augmented reality content configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions; presenting the augmented reality content item in association with the face depicted in the image; receiving user input selecting a region of the plural regions; determining a set of available makeup products corresponding to the selected region; and updating presentation of the augmented reality content item based on the set of available makeup products.
A method for aligning coordinate systems from separate augmented reality (AR) devices is described. In one aspect, the method includes generating predicted depths of a first point cloud by applying a pre-trained model to a first single image generated by a first monocular camera of a first augmented reality (AR) device, and first sparse 3D points generated by a first SLAM system at the first AR device, generating predicted depths of a second point cloud by applying the pre-trained model to a second single image generated by a second monocular camera of the second AR device, and second sparse 3D points generated by a second SLAM system at the second AR device, determining a relative pose between the first AR device and the second AR device by registering the first point cloud with the second point cloud.
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item by the person depicted in the image; receiving voice input associated with the person depicted in the image; in response to receiving the voice input, generating one or more augmented reality elements representing the voice input; and applying the one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person.
A method for dynamically initializing a 3 degrees of freedom (3DOF) tracking device is described. In one aspect, the method includes accessing a gyroscope signal from a gyroscope of the 3DOF tracking device, accessing an accelerometer signal from an accelerometer of the 3DOF tracking device, determining an initial state includes a combination of an initial orientation, an initial position, and an initial velocity of the 3DOF tracking device, the initial state indicating a starting condition of the 3DOF tracking device, integrating the gyroscope signal and the accelerometer signal to obtain orientation and position signals using the initial state, and refining an inclination signal of the orientation signal using the position signal.
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/038 - Dispositions de commande et d'interface à cet effet, p. ex. circuits d'attaque ou circuits de contrôle incorporés dans le dispositif
A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item worn by the person depicted in the image; identifying a facial expression of the user depicted in the image; and in response to identifying the facial expression, applying one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person.
A messaging system performs engagement analysis based on labels associated with content items produced by users of the messaging system. The messaging system is configured to process content items comprising images to identify elements in the images and determine labels for the images based on conditions indicating when to associate a label of the labels with an image of the images based on the elements in the image. The messaging system is further configured to associate the label with the content item, in response to determining to associate the label with the image, associating the label with the content item. The messaging system is further configured to determine engagement scores for the label based on interactions of users with the content items associated with label and adjust the engagement scores to determine trends in the labels to generate adjusted engagement scores.
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
G06F 16/58 - Recherche caractérisée par l’utilisation de métadonnées, p. ex. de métadonnées ne provenant pas du contenu ou de métadonnées générées manuellement
G06F 18/214 - Génération de motifs d'entraînementProcédés de Bootstrapping, p. ex. ”bagging” ou ”boosting”
The subject technology applies a three-dimensional (3D) effect to image data and depth data based at least in part on an augmented reality content generator. The subject technology generates a segmentation mask based at least on the image data. The subject technology performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a packed depth map based at least in part on the a depth map of the depth data. The subject technology generates, using the processor, a message including information related to the applied 3D effect, the image data, and the depth data.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
A system and method for presentation of computer vision (e.g., augmented reality, virtual reality) using user data and a user code is disclosed. A client device can detect an image feature (e.g., scannable code) in one or more images. The image feature is determined to be linked to a user account. User data from the user account can then be used to generate one or more augmented reality display elements that can be anchored to the image feature in the one or more images.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
A63F 13/00 - Jeux vidéo, c.-à-d. jeux utilisant un affichage à plusieurs dimensions généré électroniquement
A63F 13/213 - Dispositions d'entrée pour les dispositifs de jeu vidéo caractérisées par leurs capteurs, leurs finalités ou leurs types comprenant des moyens de photo-détection, p. ex. des caméras, des photodiodes ou des cellules infrarouges
A63F 13/352 - Détails des serveurs de jeu comportant des dispositions particulières de serveurs de jeu, p. ex. des serveurs régionaux connectés à un serveur national ou à plusieurs serveurs gérant les partitions de jeu
A63F 13/58 - Commande des personnages ou des objets du jeu en fonction de la progression du jeu en calculant l’état des personnages du jeu, p. ex. niveau de vigueur, de force, de motivation ou d’énergie
A63F 13/65 - Création ou modification du contenu du jeu avant ou pendant l’exécution du programme de jeu, p. ex. au moyen d’outils spécialement adaptés au développement du jeu ou d’un éditeur de niveau intégré au jeu automatiquement par des dispositifs ou des serveurs de jeu, à partir de données provenant du monde réel, p. ex. les mesures en direct dans les compétitions de course réelles
A63F 13/79 - Aspects de sécurité ou de gestion du jeu incluant des données sur les joueurs, p. ex. leurs identités, leurs comptes, leurs préférences ou leurs historiques de jeu
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
Method for receiving an input onto a graphical user interface at a client device, capturing an image frame at the client device, the image frame comprising a depiction of an object, identifying the object within the image frame, accessing media content associated with the object within a media repository in response to identifying the object, and causing presentation of the media content within the image frame at the client device.
G06F 16/40 - Recherche d’informationsStructures de bases de données à cet effetStructures de systèmes de fichiers à cet effet de données multimédia, p. ex. diaporama comprenant des données d'image et d’autres données audio
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
G06F 16/51 - IndexationStructures de données à cet effetStructures de stockage
G06F 16/538 - Présentation des résultats des requêtes
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
An eyewear device that accurately and dynamically adjusts color and brightness of a see-through display as a function of a user's eye gaze direction and eye position using a display characteristic map. The display characteristic map is indicative of display characteristics of the see-through display. Color masks are generated as a function of the display characteristic map and the user's eye gaze direction and eye position, and a processor adjusts the see-through display characteristics based on the color masks.
Systems and methods are presented that provide for receiving, at a media overlay publication system from a first client device, content to generate a media overlay, and generating the media overlay using the content received from the client device. The generated media overlay is stored in a database associated with the media overlay publication system and associated with a first characteristic of the content received from the first client device. The media overlay is provided to a second client device when a second characteristic of context data associated with the second client device correlates to the first characteristic for the media overlay, causing a display of the media overlay on a user interface of the second client device.
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
G06F 3/04842 - Sélection des objets affichés ou des éléments de texte affichés
G06F 3/04845 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs pour la transformation d’images, p. ex. glissement, rotation, agrandissement ou changement de couleur
H04L 67/52 - Services réseau spécialement adaptés à l'emplacement du terminal utilisateur
H04N 21/431 - Génération d'interfaces visuellesRendu de contenu ou données additionnelles
H04N 21/45 - Opérations de gestion réalisées par le client pour faciliter la réception de contenu ou l'interaction avec le contenu, ou pour l'administration des données liées à l'utilisateur final ou au dispositif client lui-même, p. ex. apprentissage des préférences d'utilisateurs pour recommander des films ou résolution de conflits d'ordonnancement
H04N 21/462 - Gestion de contenu ou de données additionnelles, p. ex. création d'un guide de programmes électronique maître à partir de données reçues par Internet et d'une tête de réseau ou contrôle de la complexité d'un flux vidéo en dimensionnant la résolution ou le débit en fonction des capacités du client
A method for secure virtual currency transactions between applications operating in different security domains. A first application in a first security domain receives a request from a second application in a second security domain to access a virtual currency store, where the first security domain restricts the second application from accessing user data. The first application accesses user account data containing a virtual currency balance within its secure domain and displays a virtual currency interface with multiple virtual items. Upon receiving a user's selection of a virtual item, a purchase is initiated using the virtual currency balance, the balance is adjusted accordingly, and a purchase notification is transmitted to the second application while maintaining security restrictions on user data access.
G06F 21/62 - Protection de l’accès à des données via une plate-forme, p. ex. par clés ou règles de contrôle de l’accès
G06F 3/048 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI]
G06F 9/451 - Dispositions d’exécution pour interfaces utilisateur
G06F 16/70 - Recherche d’informationsStructures de bases de données à cet effetStructures de systèmes de fichiers à cet effet de données vidéo
G06Q 30/02 - MarketingEstimation ou détermination des prixCollecte de fonds
H04L 51/04 - Messagerie en temps réel ou quasi en temps réel, p. ex. messagerie instantanée [IM]
H04L 51/046 - Interopérabilité avec d'autres applications ou services réseau
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
H04L 67/02 - Protocoles basés sur la technologie du Web, p. ex. protocole de transfert hypertexte [HTTP]
H04L 67/53 - Services réseau en utilisant des fournisseurs tiers de services
H04M 1/72436 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité avec des moyens interactifs de gestion interne des messages pour la messagerie textuelle, p. ex. services de messagerie courte [SMS] ou courriels
H04N 1/00 - Balayage, transmission ou reproduction de documents ou similaires, p. ex. transmission de fac-similésLeurs détails
49.
GENERATING USER INTERFACES DISPLAYING AUGMENTED REALITY GRAPHICS
An Augmented Reality (AR) graphics system is provided. The AR graphics system may coordinate the display of augmented reality graphics created by multiple users located in an environment. The AR graphics system may determine an alignment object located in the environment that is designated as a common origin of a real-world coordinate system that is used to determine where to display AR graphics within the environment. Additionally, a prioritization scheme is implemented to resolve conflicts between overlapping input provided by different users in order to generate a single version of AR graphics.
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06F 3/04815 - Interaction s’effectuant dans un environnement basé sur des métaphores ou des objets avec un affichage tridimensionnel, p. ex. modification du point de vue de l’utilisateur par rapport à l’environnement ou l’objet
An Augmented Reality (AR) system provides stabilization of hand-tracking input data. The AR system provides for display a user interface of an AR application. The AR system captures, using one or more cameras of the AR system, video frame tracking data of a gesture being made by a user while the user interacts with the AR user interface. The AR system generates skeletal 3D model data of a hand of the user based on the video frame tracking data that includes one or more skeletal 3D model features corresponding to recognized visual landmarks of portions of the hand of the user. The AR system generates targeting data based on the skeletal 3D model data where the targeting data identifies a virtual 3D object of the AR user interface. The AR system filters the targeting data using a targeting filter component and provides the filtered targeting data to the AR application.
In some examples, a method to present an affordance user interface element within a user interface of an interaction application includes detecting an association of a supplemental media content item with a primary media content item presented within the user interface. The supplemental media content item is identified from among a plurality of supplemental media content items supported by the interaction application. The method may include retrieving metadata related to the supplemental media content item and presenting, within the user interface, a supplementation affordance that presents the metadata. In some examples, the supplementation affordance is user selectable via the user interface to invoke a supplementation function that enables a user to apply the supplemental media content item to a further primary media content item. The supplementation function is invoked responsive to detecting a user selection of the supplementation affordance within the user interface.
G06F 3/0481 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect
G06F 3/0484 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs
Systems, methods, and computer-readable media for adding beauty products to tutorials are presented. Methods include accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the presenter applying a beauty product to a body part of the presenter. Methods further include processing the video data to identify changes to the body part of the presenter from an application of the beauty product, and responding to identifying changes to the body part of the presenter from the application of the beauty product by processing the video data to identify the beauty product. Methods further include retrieving information regarding the beauty product and causing presentation of information regarding the beauty product on a display device.
G06K 7/10 - Méthodes ou dispositions pour la lecture de supports d'enregistrement par radiation électromagnétique, p. ex. lecture optiqueMéthodes ou dispositions pour la lecture de supports d'enregistrement par radiation corpusculaire
System, methods, devices, and instructions are described for fast boot of a processor as part of camera operation. In some embodiments, in response to a camera input, a digital signal processor (DSP) of a device is booted using a first set of instructions. Capture of image sensor data is initiated using the first set of instructions at the DSP. The DSP then receives a second set of instructions and the DSP is programmed using the second set of instructions after at least a first frame of the image sensor data is stored in a memory of the device. The first frame of the image sensor data is processed using the DSP as programmed by the second set of instructions. In some embodiments, the first set of instructions includes only instructions for setting camera sensor values, and the second set of instructions includes instructions for processing raw sensor data into formatted image files.
An electronic eyewear device communicates with a backend service system via a device hub that provides an edge proxy server for a service request from the electronic eyewear device to the backend service system. The device hub provides a standardized request/response optimized schema for providing a standardized communication between the electronic eyewear device and the backend service system in response to the service request in a standardized format adapted to minimize network requests. A standardized communication is provided to at least one backend service of the backend service system, and a standardized response to the standardized service request is received from the backend service(s) and provided to the electronic eyewear device. In one configuration, the device hub may issue asynchronous requests to backend services in response to a service request from the electronic eyewear device and merge responses into a standardized response for the electronic eyewear device.
Optical devices and methods for expanding input light and outputting the expanded light include a waveguide and an input optical element to receive light incident on a first side of the waveguide. The input optical element includes an input reflective surface to reflect the received light into the waveguide. An intermediate diffractive optical element receives light in the waveguide from a first direction, and provides an expansion of the received light in a second direction perpendicular to the first direction. An output optical element includes an output reflective surface to reflect the expanded light out of the waveguide towards a viewer. The waveguide guides light along an optical path from the input optical element to the intermediate diffractive optical element and from the intermediate diffractive optical element to the output optical element.
Method starts with processor causing virtual reality (VR) interface for communication session to be displayed on first user interface of a first head-wearable apparatus and on second user interface of second head-wearable apparatus. Processor detects first touch input from first VR input device and second touch input from second VR input device. Processor monitors location of the first touch input within the first user interface and location of the second touch input within second user interface. Processor determines distance between location of the first touch input within first user interface and location on first user interface corresponding to location of second touch input within second user interface. Processor causes first and second VR input devices to generate haptic feedback response based on the distance. Haptic feedback response increases in intensity or speed as distance decreases and decreases in intensity or speed as distance increases. Other embodiments are described herein.
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0488 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels
H04L 65/1089 - Procédures en session en ajoutant des médiasProcédures en session en supprimant des médias
Methods and systems are disclosed for generating a custom sticker. In one embodiment, a messaging application implemented on a first device receives a video and input that draws a selection of a region of the video. The messaging application generates a graphical element comprising the region of the video drawn by the input and applies one or more visual effects to the graphical element to create a custom graphic. The custom graphic with the one or more visual effects is sent from the first device to a second device.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving, by a messaging application running on a device of a user, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of the user; accessing an augmented reality content item configured to generate a plurality of completed looks with respect to applying makeup to the face; presenting the augmented reality content item, including the plurality of completed looks, in association with the face depicted in the image; receiving user input selecting a completed look of the plurality of completed looks; and displaying, in response to receiving the user input, an interface with a set of makeup products associated with the selected completed look.
A lift reporting system to perform operations that include: accessing user behavior data associated with one or more machine-learned (ML) models, the ML models associated with identifiers; determining causal conversions associated with the ML models based on the user behavior data, the causal conversions comprising values; performing a comparison between the values that represents the causal conversions; determining a ranking of the ML models based on the comparison; and causing display of a graphical user interface (GUI) that includes a display of identifiers associated with ML models.
A first extended reality (XR) device and a second XR device are colocated in an environment. The first XR device captures sensory data of a wearer of the second XR device. The sensory data is used to determine a time offset between a first clock of the first XR device and a second clock of the second XR device. The first clock and the second clock are synchronized based on the time offset and a shared coordinate system is established. The shared coordinate system enables alignment of virtual content that is simultaneously presented by the first XR device and the second XR device based on the synchronization of the first clock and the second clock.
Systems, methods, devices, instructions, and media are described for generating suggestions for connections between accounts in a social media system. One embodiment involves storing connection graph information for a plurality of user accounts, and identifying, by one or more processors of the device, a first set of connection suggestions based on a first set of suggestion metrics. A second set of connection suggestions is then identified based on a second set of suggestion metrics, wherein the second set of connection suggestions and the second set of suggestion metrics are configured to obscure the first set of connection suggestions, and a set of suggested connections is generated based on the first set of connection suggestions and the second set of connection suggestions. The set of connection suggestions is then communicated to a client device method associated with the first account.
G09G 5/395 - Dispositions spécialement adaptées pour le transfert du contenu de la mémoire à mappage binaire vers l'écran
G09G 3/36 - Dispositions ou circuits de commande présentant un intérêt uniquement pour l'affichage utilisant des moyens de visualisation autres que les tubes à rayons cathodiques pour la présentation d'un ensemble de plusieurs caractères, p. ex. d'une page, en composant l'ensemble par combinaison d'éléments individuels disposés en matrice en commandant la lumière provenant d'une source indépendante utilisant des cristaux liquides
A system and a method for generating an automated GIF file generation system is described. In one aspect, the method includes accessing an animated GIF file, identifying a plurality of elements displayed in the animated GIF file, applying a variation of one or more elements to the animated GIF file, and generating a variant animated GIF file by applying the variation of the one or more elements to the animated GIF file. The system measures a trending metric of the variant animated GIF file based on a number of times the variant animated GIF file is shared on the communication platform and uses the trending metric as a feedback to generating the variant animated GIF file.
G06F 18/21 - Conception ou mise en place de systèmes ou de techniquesExtraction de caractéristiques dans l'espace des caractéristiquesSéparation aveugle de sources
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 20/30 - ScènesÉléments spécifiques à la scène dans les albums, les collections ou les contenus partagés, p. ex. des photos ou des vidéos issus des réseaux sociaux
An encapsulated waveguide system for a near eye optical display includes a first outer layer, a second outer layer, at least one waveguide substrate comprising an input area and an output area, a first spacer and a sealing element. The at least one waveguide substrate is disposed between the first and second outer layers and spaced therefrom by the first spacer. The sealing element joins edges of the first and second outer layers so as to encapsulate the at least one waveguide substrate within a cavity formed by the first and second outer layers. The formed cavity includes a first cavity between the at least one waveguide substrate and the first outer layer and a second cavity between the at least one waveguide substrate and the second outer layer.
Systems and methods for text and audio-based real-time face reenactment are provided. An example method includes receiving an input image including a body of a person, fitting a model to the body in the input image, where the model is configured to generate an output image including the body adopting a pose based on a set of pose parameters, generating, based on the input image and the model, a three-dimensional (3D) mesh of the body, generating a texture map for the 3D mesh, modifying the texture map to modify an appearance of at least a portion of the body, and generating, based on the modified texture map and the set of pose parameters, the output image of the body adopting the pose with the modified appearance.
Systems, computer readable medium and methods for autonomous drone stabilization and navigation are disclosed. Example methods include capturing an image using an image capturing device of the autonomous drone, processing the image to identify an object, and navigating the autonomous drone relative to the object to one or more waypoints. The autonomous drone navigates initially based on a relative location of the autonomous drone from the object. The autonomous drone determines a distance from the object based on an estimated size of the object and a number of pixels of an image sensor the object occupies. The autonomous drone determines a height above a ground to assist in navigation. Additionally, the autonomous drone hovers to determine a windspeed.
G05D 1/00 - Commande de la position, du cap, de l'altitude ou de l'attitude des véhicules terrestres, aquatiques, aériens ou spatiaux, p. ex. utilisant des pilotes automatiques
B64C 39/02 - Aéronefs non prévus ailleurs caractérisés par un emploi spécial
B64U 20/87 - Montage des dispositifs d’imagerie, p. ex. montage des suspensions à cardan
B64U 101/30 - Véhicules aériens sans pilote spécialement adaptés à des utilisations ou à des applications spécifiques à l’imagerie, à la photographie ou à la vidéographie
G05D 1/222 - Dispositions de commande à distance actionnées par des humains
G05D 1/229 - Données d’entrée de commande, p. ex. points de passage
G05D 1/243 - Moyens de capture de signaux provenant naturellement de l’environnement, p. ex. signaux optiques, acoustiques, gravitationnels ou magnétiques ambiants
Multiple users can simultaneously view and scroll content from a collection of content items working on separate user systems. Example methods include generating a group feed, determining, based on metadata associated with the first user and metadata associated with the second user, a first content item of the plurality of content items, and causing the first content item to be displayed on a first computing device and on a second computing device. The methods may further include accessing, from the first user or the second user, an indication of a reaction to the first content item, accessing, from the first user or the second user, an indication to scroll to a second content item of the plurality of content items, and determining, the second content item, based on the metadata of the first user, the metadata of the second user, and the reaction to the first content item.
G06Q 50/00 - Technologies de l’information et de la communication [TIC] spécialement adaptées à la mise en œuvre des procédés d’affaires d’un secteur particulier d’activité économique, p. ex. aux services d’utilité publique ou au tourisme
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/0485 - Défilement ou défilement panoramique
An audio track with vocals is played back using a device with a display screen that displays a video feed from a camera. A location of a mouth depicted in the video feed is detected. A timestamp of playback of the audio track is compared to viseme-timestamp data for the audio track to identify a viseme corresponding to the timestamp of the audio playback a viseme is positioned at the detected location of the mouth in the video feed.
The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.
G10L 15/18 - Classement ou recherche de la parole utilisant une modélisation du langage naturel
B60H 1/00 - Dispositifs de chauffage, de refroidissement ou de ventilation
B60R 16/037 - Circuits électriques ou circuits de fluides spécialement adaptés aux véhicules et non prévus ailleursAgencement des éléments des circuits électriques ou des circuits de fluides spécialement adapté aux véhicules et non prévu ailleurs électriques pour le confort des occupants
G10L 15/183 - Classement ou recherche de la parole utilisant une modélisation du langage naturel selon les contextes, p. ex. modèles de langage
G10L 15/22 - Procédures utilisées pendant le processus de reconnaissance de la parole, p. ex. dialogue homme-machine
G10L 25/87 - Détection de points discrets dans un signal de voix
Multiple users can simultaneously view and scroll content from a collection of content items working on separate user systems. Example methods include generating a group feed, determining, based on metadata associated with the first user and metadata associated with the second user, a first content item of the plurality of content items, and causing the first content item to be displayed on a first computing device and on a second computing device. The methods may further include accessing, from the first user or the second user, an indication of a reaction to the first content item, accessing, from the first user or the second user, an indication to scroll to a second content item of the plurality of content items, and determining, the second content item, based on the metadata of the first user, the metadata of the second user, and the reaction to the first content item.
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
G06F 3/0484 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] pour la commande de fonctions ou d’opérations spécifiques, p. ex. sélection ou transformation d’un objet, d’une image ou d’un élément de texte affiché, détermination d’une valeur de paramètre ou sélection d’une plage de valeurs
G06F 3/0485 - Défilement ou défilement panoramique
G06Q 50/00 - Technologies de l’information et de la communication [TIC] spécialement adaptées à la mise en œuvre des procédés d’affaires d’un secteur particulier d’activité économique, p. ex. aux services d’utilité publique ou au tourisme
The present disclosure relates to systems and methods for enhancing the interaction between users and automated agents, such as digital assistants, by employing Large Language Models (LLMs) to infer the intent of spoken language. The invention involves continuously monitoring ambient audio, converting speech to text, and utilizing LLMs to determine whether spoken language is intended for the automated agent. A structured prompt, including the converted text and specific instructions, is sent to the LLM, which is fine-tuned to process domain-specific prompts. The LLM provides a structured output in a. standardized format, indicating the user's intent. The system may involve multiple prompts to perform separate tasks, such as identifying intent and generating additional context-specific data. This approach facilitates a more natural and intuitive user experience by eliminating the need for wake words and allowing seamless conversational interaction with virtual assistants across various platforms and devices.
Users of a chat system within an interactive platform can suspend the expiration of a plurality of content items. Example methods include generating a chat, the chat comprising an association between a first user account and a second user account, and receiving, from a first user system associated with the first user account, an indication of a plurality of content items and an indication of the chat. The method may further include sending, to a second user account, the plurality of content items and an indication of the chat, and receiving, from the second user account, an indication to save the plurality of content items within the chat. The method may further include setting a saved data field associated with the plurality of content items and the second user account, the saved data field indicating the plurality of content items do not expire within the chat.
Systems, devices, media, and methods are presented for an immersive augmented reality (AR) experience using an eyewear device. A portable eyewear device includes a processor, a memory, and a display projected onto at least one lens assembly. The memory has programming stored therein that, when executed by the processor, captures information depicting an environment surrounding the device and identifies a match between objects in that information and predetermined objects in previously obtained information for the same environment. When the position of the eyewear device reaches a preselected location with respect to the matched objects, a physical output is provided to produce the immersive experience. The physical output changes as the position of the eyewear device moves to maintain the immersive experience.
A messaging server system receives a message creation input from a first client device that is associated with a first user registered with the messaging server system. The messaging server system determines, based on an entity graph representing connections between a plurality of users registered with the messaging server system, that the first user is within a threshold degree of connection with a second that initiated a group story in relation to a specified event. The messaging server system determines, based on location data received from the first client device, that the first client device was located within a geo-fence surrounding a geographic location of the specified event during a predetermined event window, the geo-fence and event window having been designated by the second user, and causes the first client device to present a user interface element that enables the first user to submit content to the group story.
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
H04L 51/222 - Surveillance ou traitement des messages en utilisant des informations de localisation géographique, p. ex. des messages transmis ou reçus à proximité d'un certain lieu ou d'une certaine zone
H04M 1/72436 - Interfaces utilisateur spécialement adaptées aux téléphones sans fil ou mobiles avec des moyens de soutien local des applications accroissant la fonctionnalité avec des moyens interactifs de gestion interne des messages pour la messagerie textuelle, p. ex. services de messagerie courte [SMS] ou courriels
H04W 4/021 - Services concernant des domaines particuliers, p. ex. services de points d’intérêt, services sur place ou géorepères
Systems, computer readable medium and methods for fully autonomous drone flight are disclosed. Example methods include taking off, navigating in accordance with a flight plan, and navigating the autonomous drone to land. The autonomous drone performs flight plans with only an initial command for the autonomous drone to fly and, in some examples, an indication of a landing space such as an open hand presented under the autonomous drone. After an initial fly command, the autonomous drone is not controlled by a remote-control device and does not receive any additional commands to complete the flight plan. The autonomous drone enters a lower energy state while flying where the wireless connections are turned off since the autonomous drone does not respond to commands during flight.
B64U 101/30 - Véhicules aériens sans pilote spécialement adaptés à des utilisations ou à des applications spécifiques à l’imagerie, à la photographie ou à la vidéographie
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 20/17 - Scènes terrestres transmises par des avions ou des drones
G06V 40/10 - Corps d’êtres humains ou d’animaux, p. ex. occupants de véhicules automobiles ou piétonsParties du corps, p. ex. mains
G06V 40/16 - Visages humains, p. ex. parties du visage, croquis ou expressions
Systems and methods for text and audio-based real-time face reenactment are provided. An example method includes receiving an input image including a body of a person, fitting a model to the body in the input image, generating a warped depth map and a warped normal map corresponding to the body in the input image, generating, based on the warped depth map and the warped normal map, a point cloud representing a surface of the body, generating, by traversing the point cloud, a first mesh for a front side surface of the body and a second mesh for a back side surface of the body, and merging the first mesh and the second mesh into a reconstructed three-dimensional mesh of the body.
In a camera-enabled electronic device, photo capture is triggered by a press-and-hold input only if the holding duration of the press-and-hold input is greater than a predefined threshold duration. A press-and-hold input shorter in duration than the threshold triggers video capture. Thus, a short press triggers video capture, while a long press triggers photo capture.
H04N 23/667 - Changement de mode de fonctionnement de la caméra, p. ex. entre les modes photo et vidéo, sport et normal ou haute et basse résolutions
G08B 5/36 - Systèmes de signalisation optique, p. ex. systèmes d'appel de personnes, indication à distance de l'occupation de sièges utilisant une transmission électriqueSystèmes de signalisation optique, p. ex. systèmes d'appel de personnes, indication à distance de l'occupation de sièges utilisant une transmission électromécanique utilisant des sources de lumière visible
An audio track with vocals is played back using a device with a display screen that displays a video feed from a camera. A location of a mouth depicted in the video feed is detected. A timestamp of playback of the audio track is compared to vi seme-timestamp datafor the audio track to identify a viseme corresponding to the timestamp of the audio playback, and a viseme is positioned at the detected location of the mouth in the video feed.
G11B 27/031 - Montage électronique de signaux d'information analogiques numérisés, p. ex. de signaux audio, vidéo
G11B 27/11 - IndexationAdressageMinutage ou synchronisationMesure de l'avancement d'une bande en utilisant une information non détectable sur le support d'enregistrement
G11B 27/10 - IndexationAdressageMinutage ou synchronisationMesure de l'avancement d'une bande
G11B 27/28 - IndexationAdressageMinutage ou synchronisationMesure de l'avancement d'une bande en utilisant une information détectable sur le support d'enregistrement en utilisant des signaux d'information enregistrés par le même procédé que pour l'enregistrement principal
H04N 21/43 - Traitement de contenu ou données additionnelles, p. ex. démultiplexage de données additionnelles d'un flux vidéo numériqueOpérations élémentaires de client, p. ex. surveillance du réseau domestique ou synchronisation de l'horloge du décodeurIntergiciel de client
H04N 21/8547 - Création de contenu impliquant des marquages temporels pour synchroniser le contenu
80.
Display screen or portion thereof with a graphical user interface
Loading and unloading of ML models into an ML model cache or system memory of an electronic eyewear device is managed based on which applications are active or available and predicted activities. Sensor inputs are processed to detect whether the electronic eyewear device has moved or is predicted to move and new ML models are downloaded based on updated location information or observable visual information. Sensor inputs are also processed to determine whether the electronic eyewear device has changed state or resource availability and whether the ML model cache or system memory needs to be resized to accommodate new ML models for the changed conditions. If so, stored ML models are updated to reflect the new device state by unloading an ML model, receiving a new ML model based on the changed state or resource availability and a processing priority of the new ML model, or both.
Systems, devices, media, and methods are presented for selectively activating and suspending control of a graphical user interface by two or more electronic devices. A portable eyewear device includes a display projected onto at least one lens assembly and a primary touchpad through which the user may access a graphical user interface (GUI) on the display. A handheld accessory device, such as a ring, includes an auxiliary touchpad that is configured to emulate the primary touchpad. The eyewear processor temporarily suspends inputs from one touchpad when it detects an activation signal from the other touchpad.
G06F 3/0354 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection des mouvements relatifs en deux dimensions [2D] entre le dispositif de pointage ou une partie agissante dudit dispositif, et un plan ou une surface, p. ex. souris 2D, boules traçantes, crayons ou palets
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06F 3/038 - Dispositions de commande et d'interface à cet effet, p. ex. circuits d'attaque ou circuits de contrôle incorporés dans le dispositif
G06F 3/04883 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] utilisant des caractéristiques spécifiques fournies par le périphérique d’entrée, p. ex. des fonctions commandées par la rotation d’une souris à deux capteurs, ou par la nature du périphérique d’entrée, p. ex. des gestes en fonction de la pression exercée enregistrée par une tablette numérique utilisant un écran tactile ou une tablette numérique, p. ex. entrée de commandes par des tracés gestuels pour l’entrée de données par calligraphie, p. ex. sous forme de gestes ou de texte
An optical waveguide is disclosed. The optical waveguide is to provide pupil expansion in two dimensions with input and output ends and having a first axis substantially parallel to the direction of propagation of light in the waveguide and substantially parallel with a direction from the input end to the output end. The optical waveguide includes an input region; a beam splitter to expand light received from the input region; and a symmetrical diffraction grating comprising complementary first and second grating portions. The second grating portion is substantially symmetrical to the first grating portion along a line of symmetry that is substantially parallel to the first axis. Light received at the diffraction grating from the beam splitter is to be diffracted by the grating towards the line of symmetry by the first or second grating portion.
Examples described herein relate to hand-based light estimation for extended reality (XR). An image sensor of an XR device is used to obtain an image of a hand in a real-world environment. At least part of the image is processed to detect a pose of the hand. One of a plurality of machine learning models is selected based on the detected pose. At least part of the image is processed via the machine learning model to obtain estimated illumination parameter values associated with the hand. The estimated illumination parameter values are used to render virtual content to be presented by the XR device.
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06T 19/00 - Transformation de modèles ou d'images tridimensionnels [3D] pour infographie
G06V 10/70 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
86.
INTER-FRAME FEATURE MAP COMPRESSION FOR STATEFUL INFERENCE
Examples described herein relate to stateful inference of a neural network. A plurality of feature map segments each has a first set of values stored in a compressed manner. The first sets of values at least partially represent an extrinsic state memory of the neural network after processing of a previous input frame. Operations are performed with respect to each feature map segment. The operations include decompressing and storing the first set of values. The operations further include updating at least a subset of the decompressed first set of values based on a current input frame to obtain a second set of values. The second set of values is compressed and stored. Memory resources used to store the decompressed first set of values is released. The second sets of values at least partially represent the extrinsic state memory of the neural network after processing of the current input frame.
Examples described herein relate to hand-based light estimation for extended reality (XR). An image sensor of an XR device is used to obtain an image of a hand in a real-world environment. At least part of the image is processed to detect a pose of the hand. One of a plurality of machine learning models is selected based on the detected pose. At least part of the image is processed via the machine learning model to obtain estimated illumination parameter values associated with the hand. The estimated illumination parameter values are used to render virtual content to be presented by the XR device.
Visual-inertial tracking of an eyewear device using a rolling shutter camera(s). The device includes a position determining system. Visual-inertial tracking is implemented by sensing motion of the device. An initial pose is obtained for a rolling shutter camera and an image of an environment is captured. The image includes feature points captured at a particular capture time. A number of poses for the rolling shutter camera is computed based on the initial pose and sensed movement of the device. The number of computed poses is responsive to the sensed movement of the mobile device. A computed pose is selected for each feature point in the image by matching the particular capture time for the feature point to the particular computed time for the computed pose. The position of the mobile device is determined within the environment using the feature points and the selected computed poses for the feature points.
Systems and methods described herein provide for retrieving, from a storage device, first image data previously captured by a client device. The systems and methods further detect a selection of a first image processing operation and perform the first image processing operation on the first image data to generate second image data. The systems and methods further detect a selection of a second image processing operation and perform the second image processing operation on the second image data to generate third image data. The systems and methods generate a message comprising the third image data.
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
90.
AUTOMATED ADJUSTMENT OF DIGITAL CAMERA IMAGE CAPTURE PARAMETERS
A portable electronic device with image capturing capabilities automatically or semi-automatically adjusts one or more image capturing parameters based on an input attribute of user engagement with a single-action haptic input mechanism. For example, the duration for which a single-action control button carried on a frame of the device is pressed automatically determines an image stabilization mode for on-board processing of captured image data. In one example, an above-threshold press duration automatically activates a less rigorous image stabilization mode, while button release before expiry of the threshold automatically activates a more rigorous photo stabilization mode.
G02C 11/00 - Accessoires non optiquesFixation de ceux-ci
H04N 5/77 - Circuits d'interface entre un appareil d'enregistrement et un autre appareil entre un appareil d'enregistrement et une caméra de télévision
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
H04N 23/667 - Changement de mode de fonctionnement de la caméra, p. ex. entre les modes photo et vidéo, sport et normal ou haute et basse résolutions
H04N 23/68 - Commande des caméras ou des modules de caméras pour une prise de vue stable de la scène, p. ex. en compensant les vibrations du boîtier de l'appareil photo
Methods and systems are disclosed for training a machine learning (ML) model to detect inner speech. The system collects, by an electromyograph (EMG) communication device used by a user, a first set of EMG signals over a first time interval. The system generates a first plurality of features based on the first set of EMG signals and generates a first probability associated with presence of inner speech by processing the first plurality of features with a machine learning (ML) model. The system compares the first probability generated by the ML model to a specified threshold and detects presence of the inner speech of the user in response to determining that the first probability generated by the ML model transgresses the specified threshold.
Systems, devices, media, and methods are presented for using a flexible electronic device to selectively interact with an eyewear device. A portable eyewear device includes a processor, a memory, and a display projected onto at least one lens assembly. A flexible electronic device includes an integrated circuit, a plurality of input sensors, and a power system, all mounted on a flexible substrate that is sized and shaped to conform to a graspable object such as a ring. The flexible electronic device operates according to a power budget, operating on a sensor power budget until it detects a first interaction with at least one of the input sensors. If the first interaction exceeds a sensitivity threshold, the flexible electronic device sends a wake signal to a nearby eyewear device. In response to the wake signal, the eyewear device presents a graphical user interface (GUI) on the display. The eyewear device further presents a cursor along a path on the display that is substantially correlated to the course traveled by the flexible electronic device in motion along a course.
Example systems, devices, media, and methods are described for presenting an interactive game in augmented reality on the display of a smart eyewear device. A hand tracking utility detects and tracks the location of hand gestures in real time, based on high-definition video data. The detected hand gestures are compared to a library of hand gestures and landmarks. Examples include synchronized, multi-player games in which each device detects and shares hand gestures with other devices for evaluation and scoring. A single-player example includes gesture-shaped icons presented on a virtual scroll that appears to move toward an apparent collision with corresponding key images, awarding points if the player's hand is located near the apparent collision and the detected hand shape matches the moving icon.
A63F 13/428 - Traitement des signaux de commande d’entrée des dispositifs de jeu vidéo, p. ex. les signaux générés par le joueur ou dérivés de l’environnement par mappage des signaux d’entrée en commandes de jeu, p. ex. mappage du déplacement d’un stylet sur un écran tactile en angle de braquage d’un véhicule virtuel incluant des signaux d’entrée de mouvement ou de position, p. ex. des signaux représentant la rotation de la manette d’entrée ou les mouvements des bras du joueur détectés par des accéléromètres ou des gyroscopes
A63F 13/213 - Dispositions d'entrée pour les dispositifs de jeu vidéo caractérisées par leurs capteurs, leurs finalités ou leurs types comprenant des moyens de photo-détection, p. ex. des caméras, des photodiodes ou des cellules infrarouges
A63F 13/26 - Dispositions de sortie pour les dispositifs de jeu vidéo ayant au moins un dispositif d’affichage supplémentaire, p. ex. sur la manette de jeu ou à l’extérieur d’une cabine de jeu
A63F 13/537 - Commande des signaux de sortie en fonction de la progression du jeu incluant des informations visuelles supplémentaires fournies à la scène de jeu, p. ex. en surimpression pour simuler un affichage tête haute [HUD] ou pour afficher une visée laser dans un jeu de tir utilisant des indicateurs, p. ex. en montrant l’état physique d’un personnage de jeu sur l’écran
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/042 - Numériseurs, p. ex. pour des écrans ou des pavés tactiles, caractérisés par les moyens de transduction par des moyens opto-électroniques
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06T 11/60 - Édition de figures et de texteCombinaison de figures ou de texte
94.
SIGN LANGUAGE INTERPRETATION WITH COLLABORATIVE AGENTS
A method for recognizing sign language using collaborative augmented reality devices is described. In one aspect, a method includes accessing a first image generated by a first augmented reality device and a second image generated by a second augmented reality device, the first image and the second image depicting a hand gesture of a user of the first augmented reality device, synchronizing the first augmented reality device with the second augmented reality device, in response to the synchronizing, distributing one or more processes of a sign language recognition system between the first and second augmented reality devices, collecting results from the one or more processes from the first and second augmented reality devices, and displaying, in near real-time in a first display of the first augmented reality device, text indicating a sign language translation of the hand gesture based on the results.
G06F 40/58 - Utilisation de traduction automatisée, p. ex. pour recherches multilingues, pour fournir aux dispositifs clients une traduction effectuée par le serveur ou pour la traduction en temps réel
G06F 3/01 - Dispositions d'entrée ou dispositions d'entrée et de sortie combinées pour l'interaction entre l'utilisateur et le calculateur
G06F 3/0346 - Dispositifs de pointage déplacés ou positionnés par l'utilisateurLeurs accessoires avec détection de l’orientation ou du mouvement libre du dispositif dans un espace en trois dimensions [3D], p. ex. souris 3D, dispositifs de pointage à six degrés de liberté [6-DOF] utilisant des capteurs gyroscopiques, accéléromètres ou d’inclinaison
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 20/20 - ScènesÉléments spécifiques à la scène dans les scènes de réalité augmentée
G06V 40/10 - Corps d’êtres humains ou d’animaux, p. ex. occupants de véhicules automobiles ou piétonsParties du corps, p. ex. mains
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
95.
GENERATING THREE-DIMENSIONAL OBJECT MODELS FROM TWO-DIMENSIONAL IMAGES
This specification discloses methods and systems for generating three-dimensional models of deformable objects from two-dimensional images. According to one aspect of this disclosure, there is described a computer implemented method for generating a three dimensional model of deformable object from a two-dimensional image. The method comprises: receiving, as input to an embedding neural network, the two-dimensional image, wherein the two dimensional image comprises an image of an object; generating, using the embedding neural network, an embedded representation of a two-dimensional image; inputting the embedded representation into a learned decoder model; and generating, using the learned decoder model, parameters of the three dimensional model of the object from the embedded representation.
Methods and systems are disclosed for performing generating AR experiences on a messaging platform. The methods and systems receive, from a client device, a request to access an augmented reality (AR) experience and access a list of event types associated with the AR experience used to generate one or more metrics. The methods and systems determine that an interaction associated with the AR experience corresponds to a first event type of the list of event types and generates interaction data for the first event type representing the interaction. In response to receiving a request to terminate the AR experience, the systems and methods transmit the interaction data to a remote server.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for rendering virtual modifications to real-world environments depicted in image content. A reference surface is detected in a three-dimensional (3D) space captured within a camera feed produced by a camera of a computing device. An image mask is applied to the reference surface within the 3D space captured within the camera feed. A visual effect is applied to the image mask corresponding to the reference surface in the 3D space. The application of the visual effect to the image mask causes a modified surface to be rendered in presenting the camera feed on a display of the computing device.
An avatar notification system is disclosed, which performs operations that include: causing display of a notification at a client device associated with a first user account, the notification including an identification of a second user account; receiving an input that selects the notification from the client device; presenting a composition interface at the client device in response to the input that selects the notification, the composition interface including a display of a media element that comprises a first identifier associated with the first user account and a second identifier associated with the second user account; receiving a selection of the media element from the client device; and generating a message that includes the media element in response to the selection.
G06F 3/04817 - Techniques d’interaction fondées sur les interfaces utilisateur graphiques [GUI] fondées sur des propriétés spécifiques de l’objet d’interaction affiché ou sur un environnement basé sur les métaphores, p. ex. interaction avec des éléments du bureau telles les fenêtres ou les icônes, ou avec l’aide d’un curseur changeant de comportement ou d’aspect utilisant des icônes
G06F 3/0482 - Interaction avec des listes d’éléments sélectionnables, p. ex. des menus
H04L 51/216 - Gestion de l'historique des conversations, p. ex. regroupement de messages dans des sessions ou des fils de conversation
H04L 51/224 - Surveillance ou traitement des messages en fournissant une notification sur les messages entrants, p. ex. des poussées de notifications des messages reçus
H04L 51/52 - Messagerie d'utilisateur à utilisateur dans des réseaux à commutation de paquets, transmise selon des protocoles de stockage et de retransmission ou en temps réel, p. ex. courriel pour la prise en charge des services des réseaux sociaux
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing an draggable shutter button during video recording. The program and method provide for displaying a user interface within an application running on a device, the user interface presenting real-time image data captured by a camera of the device, the user interface including a shutter button which is configured to be selectable by a user to initiate video recording in response to a first user gesture; and upon detecting the first user gesture selecting the shutter button, initiating video recording with respect to the real-time image data, and providing for the shutter button to be draggable in predefined directions to perform respective functions related to the video recording.
H04N 23/63 - Commande des caméras ou des modules de caméras en utilisant des viseurs électroniques
H04N 5/77 - Circuits d'interface entre un appareil d'enregistrement et un autre appareil entre un appareil d'enregistrement et une caméra de télévision
H04N 23/667 - Changement de mode de fonctionnement de la caméra, p. ex. entre les modes photo et vidéo, sport et normal ou haute et basse résolutions
Systems and methods for text and audio-based real-time face reenactment are provided. An example method includes receiving a target video that includes a target face, receiving a source video that includes a source face, determining, based on a parametric face model, facial expression parameters of the source face, modifying, in real time, the target face to imitate a face expression of the source face based on the facial expression parameters to generate a sequence of modified video frames, and displaying at least part of the sequence of modified video frames on a computing device during the generation of at least one frame of the sequence of modified video frames.
