A method for generating reference biometric data based on a bending of a flexible device is described. In one aspect, a method includes forming training data includes bending estimates of a flexible device worn by a first user, training a model based on the training data, and generating reference biometric data for the first user based on the model.
The subject technology receives at least one signal from a computing device, the at least one signal comprising at least one of a current time, battery power, sensor information, or location information. The subject technology generates a digital sticker, the digital sticker including graphical content indicating information based at least in part based on the at least one signal and media content including an image of a target face, the image of the target face being modified based on at least one of sets of source pose parameters to mimic at least one of positions of a head of a source actor and at least one of facial expressions of the source actor. The subject technology provides augmented reality content for display on a computing device, the augmented reality content including the digital sticker as an overlay on at least a portion of the augmented reality content.
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
A neural light field (NeLF) that runs real-time on mobile devices for neural rendering of three dimensional (3D) scenes, referred to as MobileR2L. The MobileR2L architecture runs efficiently on mobile devices with low latency and small size, and it achieves high-resolution generation while maintaining real-time inference for both synthetic and real-world 3D scenes on mobile devices. The MobileR2L has a network backbone including a convolutional layer embedding an input image at a resolution, residual blocks uploading the embedded image, and super-resolution modules receiving the uploaded embedded image and rendering an output image having a higher resolution than the embedded image. The convolution layer generates a number of rays equal to a number of pixels in the input image, where a partial number of the rays is uploaded to the super-resolution modules.
G06T 3/4046 - Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
G06T 3/4053 - Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
An electronic eyewear device having temples that are lightweight, aesthetically pleasant, and include thermal management. The temples each have a thermally conductive stiffener with a non-thermally conductive overmolded material forming a window. The window in each temple exposes interior components, such the stiffener to ambient air, and allows heat generated by electronic components to be released to ambient air to cool the components through convection. The windows also allow the stiffeners to be coupled to tooling and are easily overmolded. The windows provide both an aesthetic feature and a functional feature.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing shared content collections. The program and method provide for receiving, from a first device of a first user, an indication of first user input to share a content collection between the first user and a second user selected by the first user, the content collection comprising at least one media content item, the second user corresponding to a contact of the first user; storing the content collection in association with the first user and the second user; receiving an indication of second user input to share the content collection with a third user selected by the second user, the third user corresponding to a contact of the second user; and associating the content collection with the third user.
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
An environment synthesis framework generates virtual environments from a synthesized two-dimensional (2D) satellite map of a geographic area, a three-dimensional (3D) voxel environment, and a voxel-based neural rendering framework. In an example implementation, the synthesized 2D satellite map is generated by a map synthesis generative adversarial network (GAN) which is trained using sample city datasets. The multi-stage framework lifts the 2D map into a set of 3D octrees, generates an octree-based 3D voxel environment, and then converts it into a texturized 3D virtual environment using a neural rendering GAN and a set of pseudo ground truth images. The resulting 3D virtual environment is texturized, lifelike, editable, traversable in virtual reality (VR) and augmented reality (AR) experiences, and very large in scale.
A method is provided for delivering location-based information using artificial intelligence. The method includes automatically inferring a location of a user system at a geographic landmark based on location data, and triggering an AI assistant in response to the inferred location. The AI assistant generates information about the geographic landmark, and a graphical indication of the AI-generated information is displayed proximate to a graphical representation of the user on a map interface. Upon user selection of the graphical indication, a chat conversation with the AI assistant is initiated, the conversation including the AI-generated information about the geographic landmark
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program and a method for rendering a virtual object in a real-world environment depicted in image content based on contextual information. A virtual object template is selected. One or more stylizations for the virtual object template are determined based on contextual information associated with a computing device. A virtual object is generated by applying the one or more stylizations to the virtual object template. The virtual object is rendered within a 3D space captured within a camera feed of the computing device.
A modular image processing SDK comprises an API to receive API calls from third party software running on a portable device including a camera. SDK logic receives and processes commands and parameters received from the API that are based on the API calls received from the third party software. An annotation system performs image processing operations on a feed from the camera based on image processing instructions and parameters received by the annotation system from the SDK logic. The image processing is based at least in part on augmented reality content generator data (or AR content generators), user input and sensor data.
A method is provided for delivering location-based information using artificial intelligence. The method includes automatically inferring a location of a user system at a geographic landmark based on location data, and triggering an Al assistant in response to the inferred location. The Al assistant generates information about the geographic landmark, and a graphical indication of the Al-generated information is displayed proximate to a graphical representation of the user on a map interface. Upon user selection of the graphical indication, a chat conversation with the Al assistant is initiated, the conversation including the Al-generated information about the geographic landmark
Systems and methods herein described accurate and anonymized attribution. The described systems and methods access a set of impression data, access a set of conversion data, for every predefined time period, generates a salt value, generates a hashed set of impression data by appending the salt value to at least a portion of entries in the set of impression data, generates a hashed set of conversion data by appending the salt value to at least a portion of entries in the set of conversion data, discards the generated salt value, generates an intersection set of the hashed set of impression data and the hashed set of conversion data, determines a count based on the intersection set, and stores the count.
An extended Reality (XR) system is provided that monitors neurological signals to determine an engagement of a user with a real-world environment. The XR system continuously monitors neurological signals of a user through a processor operating in a low-power mode. The XR system generates an engagement signal by analyzing endogenous brain patterns in the neurological signals. In response to the engagement signal, the XR system activates environmental sensors to capture real-world environment data. The XR system generates contextual data from the captured environment data and determines XR content to provide to the user based on the contextual data. The XR system selectively activates XR capabilities to display the determined XR content.
A support arm assembly for a head-worn device provides radio frequency (RF) shielding for a projector. A metal support arm, configured to structurally attach to a rear structural element and an optical element holder of the head-worn device, forms a rear face, a bottom face, and a top face of an enclosure. A metal front face of the enclosure attaches to the optical element holder, and defines a front aperture for permitting passage of light from an exit pupil of the projector toward an input optical element. The metal support arm forms a structural support joining the optical element holder to the rear structural element without placing mechanical load on the projector. A first side face of the enclosure and a second side face of the enclosure are electrically coupled to the metal support arm.
A media curation system configured to perform operations that include, capturing an image at a client device, wherein the image includes a depiction of an object, identifying an object category of the object based on the depiction of the object within the image, accessing media content associated with the object category within a media repository, generating a presentation of the media content, and causing display of the presentation of the media content within the image at the client device.
G06F 16/48 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
The users' experience of engaging with augmented reality (AR) technology that permits users to interact with their environment and with each other is enhanced by automatically selecting an AR experience that is suitable for use given the physical environment of the user. The physical environment of the user is the physical environment of the user's computing device. The physical environment may include objects and/or conditions present in close proximity to the user's computing device, such as other humans, animals, and smart devices.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
G06T 11/60 - Editing figures and textCombining figures or text
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
H04L 67/131 - Protocols for games, networked simulations or virtual reality
H04L 67/52 - Network services specially adapted for the location of the user terminal
16.
LOCATION MAPPING FOR LARGE SCALE AUGMENTED-REALITY
An Augmented-Reality which performs operations that include: accessing a data object that comprises image data, location data, and orientation data; applying a transformation to the data object to produce a rectified data object; generating a point cloud based on the rectified data object; assigning the point cloud to a location based on at least the location data of the data object; detecting a client device at the location; and loading the point cloud to the client device in response to the detecting the client device at the location.
Systems and methods are provided. For example, a method includes determining a position of a user's hand and identifying a manipulation gesture performed by the user targeting a virtual object. The method also includes determining a three-dimensional (3D) origin point based on the position of the user's hand when the manipulation gesture is performed, and determining a 3D end point based on a movement of the user's hand from the origin point. The method additionally includes deriving a 3D vector based on the 3D origin point and the 3D end point, and applying an action to the targeted virtual object based on the 3D vector, wherein the targeted virtual object is at a distance greater than the user's arm reach.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/03 - Arrangements for converting the position or the displacement of a member into a coded form
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/0487 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
G06V 10/70 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning
G06V 40/20 - Movements or behaviour, e.g. gesture recognition
18.
QUOTABLE STORIES AND STICKERS FOR MESSAGING APPLICATIONS
A system includes one or more hardware processors and at least one memory storing instructions that cause the one or more hardware processors to perform operations including receiving, via a client device, one or more response messages to a question sticker or to a story, and selecting, via the client device, a response message of the one or more response messages for publication. The operations also include selecting, via the client device, a privacy setting for the response message, and publishing, via the client device, the response message.
H04L 51/02 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
19.
REQUEST QUEUE FOR SHARED CONTROL OF CAMERA DEVICE BY MULTIPLE DEVICES
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, method, and user interface to facilitate a camera sharing session between two or more users. A camera sharing session is initiated based on session configuration information comprising a user identifier of a user permitted to control image capturing at a camera communicatively coupled to a first device. A trigger request is received from the second device and in response, an image capture, which results in at least one image, is triggered at the camera and the image is transmitted to the second device.
Method of generating modified media content item based on user's setting selection starts with a processor causing a camera personalization interface to be displayed by a display of a client device. The processor receives a setting selection from the client device via the camera personalization interface. The processor determines a pre-capture setting and a post-processing setting based on the setting selection. The processor calibrates a camera of the client device using the pre-capture setting. The processor receives a media content item including an image captured using the camera and generates a modified media content item by modifying the media content item using the post-processing setting. Other embodiments are described herein.
The present invention relates to a messaging system configured to: receive a request to generate a message at a first client device; cause display of a message notification within an interface of a second client device, wherein the message was addressed to a recipient of the second client device; receive a request to un-send the message from the first client device; and remove the message notification from the interface at the second client device in response to the request to un-send the message, according to some example embodiments.
Systems and methods herein describe a method for capturing a video in real-time by an image capture device. The system provides a plurality of visual pose hints, identifies first pose information in the video while capturing the video, applies a first series of virtual effects to the video, identifies second pose information, and applies a second series of virtual effects to the video, the second series of virtual effects based on the first series of virtual effects.
A social network image pool system can capture one or more image data items (e.g., image, video) in a temporary persistent post pool. The post pool enables for efficient capture of multiple image data items for publishing in a manner that allows multiple images data items to be captured while preserving the editability of the multiple items before they are published to a social network site.
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
H04L 67/143 - Termination or inactivation of sessions, e.g. event-controlled end of session
Methods and systems are disclosed for generating an extended reality (XR) try-on experience based on an image produced by a diffusion model. The system receives an image depicting a real-world object and generates a prompt comprising a textual description of a fashion item. The system analyzes the image and the textual description of the fashion item using a generative machine learning model to generate an artificial image that depicts an artificial object that resembles the real-world object wearing an artificial fashion item matching the textual description of the fashion item. The system identifies an object comprising a real-world product image that matches visual attributes of the artificial fashion item and replaces the artificial fashion item in the artificial image with the object to generate an output image.
Systems and methods are provided for using an external controller with an AR device. The system establishes, by one or more processors of the AR device, a communication with an external client device. The system overlays, by the AR device, a first AR object on a real-world environment being viewed using the AR device. The system receives interaction data from the external client device representing one or more inputs received by the external client device and, in response, modifies the first AR object by the AR device.
A63F 13/655 - Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor automatically by game devices or servers from real world data, e.g. measurement in live racing competition by importing photos, e.g. of the player
A63F 13/211 - Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
A63F 13/533 - Controlling the output signals based on the game progress involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game for prompting the player, e.g. by displaying a game menu
A63F 13/573 - Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game using trajectories of game objects, e.g. of a golf ball according to the point of impact
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
An XR system is provided that enhances user interaction within extended reality environments through precise hand scale estimation. The XR system is configured to capture tracking data of a user's hand as the user interacts with a mobile device. Concurrently, the XR system captures pose data of itself and uses the tracking data and the pose data to determine a reference line segment. This segment aids in calculating three-dimensional distances between node pairs of the user's hand. By employing these measurements, the XR system effectively calculates a hand scale factor that is used for accurately integrating the user's hands into an XR user interface.
An XR system is provided. This system captures images including images of a first hand of a user and a second hand of the user using one or more cameras. The XR system generates cropped images using the images, each cropped image including a surface of the first hand. The XR system detects a hand touch of the surface of the hand by a digit of the second hand using the cropped images. The hand touch is used as an input into an XR user interface of the XR system. The surface of the hand can be palmar surface or a hand dorsal surface.
An XR system is provided that enhances user interaction within extended reality environments through precise hand scale estimation. The XR system is configured to capture tracking data of a user's hand as the user interacts with a mobile device. Concurrently, the XR system captures pose data of itself and uses the tracking data and the pose data to determine a reference line segment. This segment aids in calculating three-dimensional distances between node pairs of the user's hand. By employing these measurements, the XR system effectively calculates a hand scale factor that is used for accurately integrating the user's hands into an XR user interface.
G06F 3/0488 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
29.
SYSTEMS AND METHODS FOR LOW POWER COMMON ELECTRODE VOLTAGE GENERATION FOR DISPLAYS
A system, circuit, and method for implementing a low power common electrode voltage for a display (e.g., LcoS display) having transistors with low to moderate breakdown voltages may include a first and a second low voltage amplifier, wherein the first amplifier generates a pixel voltage and the second amplifier generates a predetermined voltage. The circuit may include a common electrode circuit coupled to the first and second amplifier to generate a common electrode voltage. Particularly, the circuit may include a control circuit coupled to the common electrode circuit, wherein, during a first phase, the control circuit selectively controls the common electrode circuit to generate a low common electrode voltage based upon a negative value of the predetermined voltage. Further, during a second phase, the control circuit selectively controls the common electrode circuit to generate a high common electrode voltage based upon the sum of the predetermined voltage and the pixel voltage.
G09G 3/36 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix by control of light from an independent source using liquid crystals
30.
BEAUTIFICATION TECHNIQUES FOR 3D DATA IN A MESSAGING SYSTEM
The subject technology applies, to image data and depth data, a 3D effect including at least one beautification operation based on an augmented reality content generator, the 3D effect including a beautification operation, the beautification operation comprising modifying image data, the image data including a region corresponding to a representation of a face, the beautification operation comprising using a machine learning model for at least one of smoothing blemishes or preserving facial skin texture. The subject technology generates a depth map using at least the depth data. The subject technology generates a segmentation mask based at least on the image data. The subject technology performs background inpainting and blurring of the image data using at least the segmentation mask to generate background inpainted image data. The subject technology generates a 3D message based at least in part on the applied 3D effect including the at least one beautification operation.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 3/04883 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Systems and methods are provided for performing operations including: receiving, via a messaging application of a user device, a request to recover access to an account of a user of the messaging application; accessing a first object corresponding to a first key; receiving, from a first friend of the user on the messaging application, a second object corresponding to a first portion of a second key; receiving, from a second friend of the user on the messaging application, a third object corresponding to a second portion of the second key; deriving the second key based on the second and third objects; and recovering access to the account of the user based on the first key and the second key.
Methods and systems are disclosed for sharing collections of content items in chat sessions. The methods and systems receive a request to share a first content item and present a GUI comprising a first set of options and a second set of options, the first set of options being associated with adding the first content item to a collection of content items that is accessible to a plurality of recipients, the second set of options being associated with sending the first content item to individual recipients. The methods and systems determine a set of target recipients of the first content item and select a content sharing link between a first link to the collection of content items and a second link directly to the first content item. The methods and systems send, to a target recipient, the content sharing link that has been selected.
H04N 21/4788 - Supplemental services, e.g. displaying phone caller identification or shopping application communicating with other users, e.g. chatting
H04N 21/231 - Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers or prioritizing data for deletion
H04N 21/431 - Generation of visual interfacesContent or additional data rendering
33.
SELECTIVE COLLABORATIVE OBJECT ACCESS BASED ON TIMESTAMP
Collaborative sessions in which access to added virtual content is selectively made available to participants/users by a collaborative system. The system receives a request from a user to join a session, and associates a timestamp with the user corresponding to receipt of the request. Users can edit the collaborative object if the timestamp is within the collaborative duration period and can view the collaborative object if the timestamp is after the collaborative duration period.
A device that measures a size of a user's face, referred to as face scaling, using a monocular camera. Depth is calculated from sparse feature points. A face mesh is used to improve the estimation accuracy. A processing pipeline detects face features by applying a face landmark detection algorithm to find the important face feature points such as the eyes, nose, and mouth. The processing pipeline estimates feature points depth using depth obtained through image defocus. The processing pipeline further scales the face using an estimated depth of the face features.
An XR system is provided. This system captures images including images of a first hand of a user and a second hand of the user using one or more cameras. The XR system generates cropped images using the images, each cropped image including a surface of the first hand. The XR system detects a hand touch of the surface of the hand by a digit of the second hand using the cropped images. The hand touch is used as an input into an XR user interface of the XR system. The surface of the hand can be palmar surface or a hand dorsal surface.
The present disclosure provides a method for creating a 3D model of a reference surface. The method includes capturing, using a capture device, a plurality of data points on the reference surface, determining a position and an orientation of the capture device related to the capture of the plurality of data points, creating a 3D data representation of the reference surface based on the plurality of data points, creating a location tracking data representation of the reference surface based the plurality of data points on the reference surface and the position and the orientation of the capture device, and creating the 3D model of the reference surface based on the 3D data representation and the location tracking data representation of the reference surface.
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for modifying a captured image. The program and method provide for displaying, by a messaging application, an image captured by a device camera; providing, by the messaging application, a user interface for selecting from among a plurality of content modifiers to modify the image, the plurality of content modifiers including a first content modifier corresponding to a machine learning model trained with a plurality of image pairs, each image pair including a first image and a second image corresponding to a modified version of the first image; receiving user selection of the first content modifier from among the plurality of content modifiers; determining, in response to receiving the user selection, a modified version of the image based on output from the machine learning model; and displaying the modified version of the image.
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
38.
EYEWEAR HAVING CURRENT CONSUMPTION OPTIMIZATION OF WIRELESS SYSTEM INTERFACE
Eyewear having a high-speed wireless transceiver, including a processor having a high-speed interface for communicating high-speed communications and a low-speed interface for communicating low-speed communications. The high-speed interface is disabled to have no standby current and only the low-speed interface is used when only low-speed communications are needed to save current. When communications are received via the high-speed wireless transceiver, only the low-speed interface is initially used, and the high-power interface is later used if necessary. The high-speed interface can be a high-speed universal serial bus (USB) interface, and the low-speed interface can be a universal asynchronous receiver-transmitter (UART) interface. A USB hub is controlled by the processor to selectively enable the USB interface.
A system is disclosed, including a display, a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Active region data is generated that includes, for each of one or more active regions, active region location data and active region content. The active region data is transmitted to a display having a display area. For each active region, the active region content is displayed at an active region location of the display area based on the active region location data, the active region content being displayed at a higher spatiotemporal information density than content displayed in the display area outside of the active regions.
Methods and systems are disclosed for transferring garments from a real-world object to a virtual object. The system receives, by a client device, an image that includes a depiction of a real-world object having a fashion item in a real-world environment. The system accesses a three-dimensional (3D) avatar model of a human and generates a graphic item corresponding to the fashion item being worn by the real-world object depicted in the image. The system modifies the 3D avatar model of the human based on the graphic item and presents the 3D avatar model that has been modified based on the graphic item within a view of the real-world environment on the client device.
A finger gesture recognition system is provided. The finger gesture recognition system includes one or more audio sensors and one or more optic sensors. The finger gesture recognition system captures, using the one or more audio sensors, audio signal data of a finger gesture being made by a user, and captures, using the one or more optic sensors, optic signal data of the finger gesture. The finger gesture recognition system recognizes the finger gesture based on the audio signal data and the optic signal data and communicates finger gesture data of the recognized finger gesture to an Augmented Reality/Combined Reality/Virtual Reality (XR) application.
An approach for publishing posts on a social network through one or more user accounts with different levels of attribution is disclosed. A secure user account publishes a post through a programmatically linked buffer user account. The secure user account and the buffer user account are programmatically linked. Posts published via the buffer user account can be modified to add attribution image data or other visual indicators of the original post creator.
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
A method for enhancing a presentation of a network document by a client terminal with real time social media content. The method comprises analyzing a content in a web document to identify a relation to a first of a plurality of multi participant events documented in an event dataset, each of the plurality of multi participant events is held in a geographical venue which hosts an audience of a plurality of participants, matching a plurality of event indicating lags of each of a plurality of user uploaded media content files with at least one feature of the first multi participant event to identify a group of user uploaded media content files selected from the plurality of user uploaded media content files, and forwarding at least some members of the group to a simultaneous presentation on a browser running on a client terminal and presenting the web document.
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
H04L 67/02 - Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
Aspects of the present disclosure involve a system comprising a storage medium storing a program and method for providing a template for media content generation. The program and method provide for receiving, from a first device of a first user, an indication of first user input setting properties to create a template for combining user-selected media with an audio track, the properties specifying the audio track, a sequence of media slots, a duration for each media slot, and predefined edits for applying to the media slots; causing display of a user interface on a second device of a second user, the user interface for assigning a respective video or photo to each media slot; and receiving, from the second device, a media content item generated based on second user input provided via the user interface, the second user input assigning the respective video or photo to each media slot.
The subject technology receives image data including a representation of a physical item. The subject technology analyzes the image data to determine an object corresponding to the physical item. The subject technology identifies a set of colors corresponding to a set of regions of the determined object. The subject technology analyzes second image data to detect a second object corresponding to a representation of a particular body part of a user. The subject technology generates augmented reality content based at least in part on the identified set of colors and the detected second object. The subject technology causes display, at a client device, the augmented reality content applied to the detected second object.
A head-wearable apparatus includes a frame having a front piece configured to hold left and right lenses. A left temple is coupled to the front piece and a right temple coupled to the front piece. A camera system includes one or more cameras coupled to the front piece, one or more left peripheral cameras coupled to an outside surface of the frame, and one or more right peripheral cameras coupled to an outside surface of the frame. A left peripheral display is coupled to an inside surface of the frame. The left peripheral display is configured to receive and display input from the one or more left peripheral cameras. A right peripheral display is coupled to an inside surface of the frame. The right peripheral display is configured to receive and display input from the one or more right peripheral cameras.
The subject technology detects a location and a position of a representation of a finger in a set of frames captured by a camera of a client device. The subject technology generates a first virtual object based at least in part on the location and the position of the representation of the finger. The subject technology renders the first virtual object within a first scene. The subject technology detects a first collision event corresponding to a first collider of the first virtual object intersecting with a second collider of a second virtual object. The subject technology modifies a set of dimensions of the second virtual object to a second set of dimensions. The subject technology renders the second virtual object based on the second set of dimensions within a second scene. The subject technology provides for display the rendered second virtual object within the second scene.
AR-enabled wearable electronic devices such as smart glasses are adapted for use as an (Internet of Things) IoT remote control device where the user can control a pointer on a television screen, computer screen, or other IoT enabled device to select items by looking at them and making selections using gestures. Built-in six-degrees-of-freedom (6DoF) tracking capabilities are used to move the pointer on the screen to facilitate navigation. The display screen is tracked in real-world coordinates to determine the point of intersection of the user's view with the screen using raycasting techniques. Hand and head gesture detection are used to allow the user to execute a variety of control actions by performing different gestures. The techniques are particularly useful for smart displays that offer AR-enhanced content that can be viewed in the displays of the AR-enabled wearable electronic devices.
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
Described is a system for generation location visualization on a map interface by identifying a current location of a user that is initiating an interaction function of an interaction client; identifying a map corresponding to the current location of the user; identifying one or more map tiles associated with the map; receiving historical location data of the user that is associated with the current location of the user; converting the historical location data into an overall polygon that is comprised of a plurality of polygons based on the identified one or more map tiles; and displaying the map with the plurality of polygons on a user interface.
Various embodiments provide for systems, methods, and computer-readable storage media that improve media content search functionality and curation of media content. For instance, various embodiments described in this document provide features that can present media content items in the form of dynamic collection of media content items upon a user typing into a search bar. In another instance, various embodiments described herein improve media content search functionality by ranking user facing search features using input signals.
G06F 16/14 - Details of searching files based on file metadata
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
Systems, methods, and computer readable media for an augmented reality (AR) shared screen space. Examples relate to a host augmented realty (AR) device sharing a screen and a relative location of the AR device to the screen with guest AR devices where the guest AR devices share a relative location of the guest AR devices to a copy of the screen displayed on the display of the guest AR devices and where the users of the AR devices may see each other's location with the use of avatars around the shared screen and add augmentations to the shared screen. The yaw, roll, and pitch of the head of the avatars tracks the movement of the head of the user of the AR wearable device.
An system for augmenting images using hand surface normal estimation is provided. In a model training phase, 3D models of hands are generated using 3D data of hands in a variety of positions. Target normal training data is generated that includes normals of surfaces of the 3D models and synthetic 2D image training data corresponding to the 3D models and the normals. The target normal training data and the synthetic image training data are used to train a normal estimation model. The normal estimation is used by an interactive application to generate augmentations that are applied to hand image data.
Examples describe adaptive image processing for an augmented reality (AR) device. An input image is captured by a camera of the AR device, and a region of interest of the input image is determined. The region of interest is associated with an object that is being tracked using an object tracking system. A crop-and-scale order of an image processing operation directed at the region of interest is determined for the input image. One or more object tracking parameters may be used to determine the crop-and-scale order. The crop-and-scale order is dynamically adjustable between a first order and a second order. An output image is generated from the input image by performing the image processing operation according to the determined crop-and-scale order for the particular input image. The output image can be accessed by the object tracking system to track the object.
A system is disclosed, including a display, a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Active region data is generated that includes, for each of one or more active regions, active region location data and active region content. The active region data is transmitted to a display having a display area. For each active region, the active region content is displayed at an active region location of the display area based on the active region location data, the active region content being displayed at a higher spatiotemporal information density than content displayed in the display area outside of the active regions.
G09G 5/02 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the way in which colour is displayed
G09G 5/38 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of individual graphic patterns using a bit-mapped memory with means for controlling the display position
An eyewear device having a frame formed of an infrared (IR) transmissive material in a continuous piece without discrete openings. IR cameras are placed behind areas of the frame designated as optical surfaces. In an example, the optical surfaces are positioned at lower corners of the frame and are angled inwardly to prevent obstructions, e.g., due to scratches and smudges. The optical surfaces have uniform thickness, are smooth, and are angled such that IR light communicated to the cameras pass at an angle through the optical surfaces. The IR cameras are angled downwardly to view objects forward and below a user wearing the eyewear device, e.g., to detect hand gestures.
A terraced battery system is provided that may enhance the battery packaging efficiency within an organic shape of a wearable electronic device such as an electronic eyewear device. The terraced battery includes several stacked cells of different geometries. The terraced battery geometries are selected to better accommodate organic (non-trapezoidal and non-cylindrical) shapes of the battery housing in the wearable electronic device. In an example, the terraced battery geometry is adapted to accommodate the organic shape of a battery housing in the temples of an augmented reality electronic eyewear device. As the number of the battery cells or terraces increases, the battery packaging efficiency can be further improved within an organic shape of the battery housing. The increased packaging efficiency for the battery enables increased battery life within organically shaped enclosures.
Systems and methods are provided for performing operations comprising: displaying a plurality of augmented reality painting options; detecting, by a touch input interface of the eyewear device, a first touch input comprising a single finger touching the touch input interface; selecting a first augmented reality painting option of the plurality of augmented reality painting options in response to the first touch input; while continuing to detect continuous touch between the single finger and the touch input interface following selection of the first augmented reality painting option, displaying a second augmented reality painting option related to the first augmented reality painting option; and performing a selection associated with the second augmented reality painting option in response to detecting, by the touch input interface, movement of the single finger along the touch input interface while continuing to detect the continuous touch.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/04815 - Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06F 3/04883 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
An input video item that includes a target visual augmentation is accessed. A machine learning model uses the input video item to generate an embedding. The embedding may comprise a vector representation of a visual effect of the target visual augmentation. The machine learning model is trained, in an unsupervised training phase, to minimize loss between training video representations generated within each of a plurality of training sets. Each training set comprises a plurality of different training video items that each include a predefined visual augmentation. Based on the generation of the embedding of the input video item, the target visual augmentation is mapped to an augmentation identifier.
Systems and methods are provided. A system includes a display and camera. The system additionally includes a secure data vault system. The secure data vault system includes a sandbox system operatively coupled to the camera and configured to receive camera data from the camera, wherein in operation of the sandbox system, the camera only sends camera data to the sandbox system, and wherein the sandbox system comprises an execution environment configured to restrict execution of instructions to a predefined memory address range. The secure data vault system additionally includes a display and rendering system operatively coupled to the sandbox system and configured to render an image based on the camera data processed via the instructions and to display the image via the display, wherein the display and rendering system is configured to blur sections of the image based on private information derived from the image.
G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
An augmented reality (AR) system includes a display, a camera; and a secure data vault system. The secure data vault system includes a sandbox system operatively coupled to the camera and configured to receive camera data from the camera, wherein in operations of the AR system, the camera only sends camera data to the sandbox system, and wherein the sandbox system comprises an execution environment configured to restrict execution of instructions to a predefined memory address range. The secure data vault system additionally includes a display and rending system operatively coupled to the sandbox system and configured to render an image based on the camera data processed via the instructions and to display the image via the display, wherein the display is configured to show both the image and a real-world environment surrounding the AR system.
G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
G06F 21/57 - Certifying or maintaining trusted computer platforms, e.g. secure boots or power-downs, version controls, system software checks, secure updates or assessing vulnerabilities
G06F 21/62 - Protecting access to data via a platform, e.g. using keys or access control rules
62.
CONTEXTUAL VISUAL AND VOICE SEARCH FROM ELECTRONIC EYEWEAR DEVICE
Augmented reality features are selected for presentation to a display of an electronic eyewear device by using a camera of the electronic eyewear device to capture a scan image and processing the scan image to extract contextual signals. Simultaneously, voice data from the user is captured by a microphone of the electronic eyewear device and voice-to-text conversion of the captured voice data is performed to identify keywords in the voice data. The extracted contextual signals and the identified keywords are then used to select at least one augmented reality feature that matches the extracted contextual signals and the identified keywords, and the selected augmented reality feature is presented to the display for user selection. The contextual information thus refines the search results to provide the augmented reality feature best suited for the context of the scan image captured by the electronic eyewear device.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
A method for programming a robotic system by demonstration is described. In one aspect, the method includes displaying a first virtual object in a display of an augmented reality (AR) device, the first virtual object corresponding to a first physical object in a physical environment of the AR device, tracking, using the AR device, a manipulation of the first virtual object by a user of the AR device, identifying an initial state and a final state of the first virtual object based on the tracking, the initial state corresponding to an initial pose of the first virtual object, the final state corresponding to a final pose of the first virtual object, and programming by demonstration a robotic system using the tracking of the manipulation of the first virtual object, the first initial state of the first virtual object, and the final state of the first virtual object.
A button-switch assembly provides a preloaded force design with an enhanced tactile feel while also providing a non-wobbly (stabilized) configuration and water/dust protection functions. Features of the button-switch assembly include excellent tactile feel through a stack up of a soft rubber layer of a deflection web and a hard PET film shim layer, a consistent pre-loaded push force through use of an angled deflection web, a button flange that minimizes rotation of the button while providing a consistent tactile feel even when the edge of the button is depressed, double sided sealing adhesive layers that seal off the opening in the housing for accepting the button to prevent water/dust from entering the opening, and gluing the button to the rubber deflection web in variable thicknesses to provide a stable tension force to minimize wobble of the button when depressed.
H01H 13/06 - Dustproof, splashproof, drip-proof, waterproof, or flameproof casings
H01H 13/705 - Switches having rectilinearly-movable operating part or parts adapted for pushing or pulling in one direction only, e.g. push-button switch having a plurality of operating members associated with different sets of contacts, e.g. keyboard with contacts carried by or formed from layers in a multilayer structure, e.g. membrane switches characterised by construction, mounting or arrangement of operating parts, e.g. push-buttons or keys
65.
Augmented reality (AR) content-sharing with AR-deprived audience
A head-worn device system equipped with cameras, display devices, and processors uses stored instructions to facilitate interactions between an augmented reality (AR) device and an external display system. When executed, these instructions establish a communications link between the AR device and the external display. The system identifies the pose of the external display and receives user inputs from the AR device user, relating to interactions with virtual objects in a real-world setting. It also identifies the positions of viewers watching the external display. Based on the external display's pose and the viewer locations, the system generates display data for the virtual object, ensuring it appears correctly on the external display. This display data is then transmitted to the external display system, completing the interaction loop and enhancing the viewing experience for the audience.
A social media platform provides a map-based graphical user interface (GUI) for accessing social media content submitted for public accessibility via the social media platform supported by the map-based GUI. The GUI includes a map providing interactive location-based searching functionality in that selection of a target location by the user in the GUI, such as by tapping or clicking at the target location, triggers a search for social media content having geo-tag data indicating geographic locations within a geographical search area centered on the target location. A search period for which content is returned is dynamically variable based on the duration for which the tap or click is held.
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/0487 - Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
Zoom and tilt operations are performed on a map display based on a predetermined relationship between zooming and tilting that is defined by a plurality of zoom-tilt transition points. If user zoom input is received after a user has tilted the map representation to a user-selected tilt angle, a next zoom-tilt transition point in the specified relationship, in the zoom direction, is determined. The display of the map representation is then updated at a rate of tilt and zoom that is based at least in part on the user zoom input, and the difference between the next zoom-tilt transition point and the initial zoom level and the user-selected tilt angle.
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
Described is a system for generating a textual response from a received image by determining participation in an interaction function by a first user of an interaction system, identifying an image associated with the participation, processing data associated with the image using a first machine learning model to identify one or more features within the image, and generating a prompt based on the identified one or more features. The system then identifying instructions for a second machine learning model, processing the prompt and the instructions using the second machine learning model to generate a textual response to the image, and causing display of the textual response within the interaction function to the first user.
G06F 40/40 - Processing or translation of natural language
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06V 10/774 - Generating sets of training patternsBootstrap methods, e.g. bagging or boosting
G06V 20/40 - ScenesScene-specific elements in video content
Methods and systems are disclosed for iteratively training a user and a ML model to produce accurate inner speech outputs. The methods and systems access a ML model and perform a first training iteration in which EMG data corresponding to inner speech is processed by the machine learning model to decode the EMG data into a set of predicted phonemes, phoneme sounds, words or phrases. The methods and systems present the set of predicted phonemes, phoneme sounds, words or phrases to the user and form a first set of training data comprising the set of predicted phonemes, phoneme sounds, words or phrases, the EMG data, and the set of specified phonemes, phoneme sounds, words or phrases as ground truth information. The methods and systems update parameters of the ML model based on the first set of training data prior to starting a second training iteration.
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Surface plane information is obtained, defining a surface plane passing through a surface location and oriented according to a surface normal. An edge is detected in an image. Virtual content is presented, having a virtual position based on an orientation of the edge and the surface plane information.
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
G06T 5/20 - Image enhancement or restoration using local operators
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/74 - Image or video pattern matchingProximity measures in feature spaces
A multi-System on Chip (SoC) hand-tracking platform is provided. The multi-SoC hand-tracking platform includes a computer vision SoC and one or more application SoCs. The computer vision SoC hosts a hand-tracking input pipeline. The one or more application SoCs host one or more applications that are consumers of input event data generated by the hand-tracking input pipeline. The applications communicate with some components of the hand-tracking input pipeline using a shared-memory buffer and with some of the components of the hand-tracking input pipeline using Inter-Process Communication (IPC) method calls.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/94 - Hardware or software architectures specially adapted for image or video understanding
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G06V 40/20 - Movements or behaviour, e.g. gesture recognition
A light emitting diode (LED) pixel array and method of fabrication thereof. A semiconductor wafer template includes a successively stacked lower n-GaN layer, lower MQW layer, lower p-GaN layer, upper n-GaN layer, and dielectric layer. A plurality of apertures is formed through the dielectric layer, extending to the upper n-GaN layer. A plurality of mesas is formed by forming, within each aperture, a mesa n-GaN layer, a mesa MQW layer above each mesa n-GaN layer, and a mesa p-GaN layer above each mesa MQW layer. The mesa n-GaN layer, mesa MQW layer, and mesa p-GaN layer of each mesa form a respective mesa LED. The lower n-GaN layer, lower MQW layer, and lower p-GaN layer form a lower LED.
H10H 20/812 - Bodies having quantum effect structures or superlattices, e.g. tunnel junctions within the light-emitting regions, e.g. having quantum confinement structures
H10H 20/821 - Bodies characterised by their shape, e.g. curved or truncated substrates of the light-emitting regions, e.g. non-planar junctions
H10H 20/825 - Materials of the light-emitting regions comprising only Group III-V materials, e.g. GaP containing nitrogen, e.g. GaN
H10H 20/832 - Electrodes characterised by their material
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: accessing, by a first application implemented on a client device, data collected from one or more entropy sources; causing a second application implemented on the client device to access the data collected from the one or more entropy sources; generating a shared cryptographic key using the data collected from one or more entropy sources; establishing a communication channel between the first application and the second application; and exchanging, over the communication channel between the first application and the second application, one or more messages that have been encrypted using the shared cryptographic key.
Systems, devices, methods, media, and instructions for automated image processing and content curation are described. In one embodiment a server computer system communicates at least a portion of a first content collection to a first client device, and receives a first selection communication in response, the first selection communication identifying a first piece of content of the first plurality of pieces of content. The server analyzes analyzing the first piece of content to identify a set of context values for the first piece of content, and accesses accessing a second content collection comprising pieces of content sharing at least a portion of the set of context values of the first piece of content. In various embodiments, different content values, image processing operations, and content selection operations are used to curate the content collections.
G06F 16/583 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
Methods and devices for wired charging and communication with a wearable device are described. In one embodiment, a symmetrical contact interface comprises a first contact pad and a second contact pad, and particular wired circuitry is coupled to the first and second contact pad to enable charging as well as receive and transmit communications via the contact pads as part of various device states.
H01R 13/62 - Means for facilitating engagement or disengagement of coupling parts or for holding them in engagement
H02J 7/04 - Regulation of the charging current or voltage
H02J 7/34 - Parallel operation in networks using both storage and other DC sources, e.g. providing buffering
H03K 19/0185 - Coupling arrangementsInterface arrangements using field-effect transistors only
H04B 3/54 - Systems for transmission via power distribution lines
H04B 3/56 - Circuits for coupling, blocking, or by-passing of signals
H10D 89/60 - Integrated devices comprising arrangements for electrical or thermal protection, e.g. protection circuits against electrostatic discharge [ESD]
76.
TECHNIQUES FOR USING 3-D AVATARS IN AUGMENTED REALITY MESSAGING
Described herein is a messaging application that executes on a wearable augmented reality device. The messaging application facilitates the anchoring or pinning of a 3-D avatar representing another end-user. An end-user wearing the AR device facilitates messaging with the other end-user via interactions with the 3-D avatar representing the other end-user. As such, the AR device processes various sensor inputs to detect when the end-user wearing the AR device is “targeting” the 3-D avatar, and enables an audio recording device to record an audible message for communicating to the other end-user.
A method for recalibrating an augmented reality (AR) device includes generating and storing a ground truth map of a real-world environment when the AR device is operating with a high likelihood of having an accurate factory calibration. During operation of the AR device, new map data is generated for the real-world environment. The new map data is compared to the ground truth map to detect potential calibration errors. If calibration errors are detected, a recalibration procedure is executed by determining an optimal path through the real-world environment that allows for observing parameters requiring recalibration. Visual cues are generated to guide a user of the AR device through the optimal path. As the user follows the visual cues, calibration parameters are iteratively adjusted to eliminate detected calibration errors. The recalibration procedure may be presented as an interactive game to improve user engagement, with rewards provided for accurately following guidance.
G01C 25/00 - Manufacturing, calibrating, cleaning, or repairing instruments or devices referred to in the other groups of this subclass
G01C 21/00 - NavigationNavigational instruments not provided for in groups
G01C 21/16 - NavigationNavigational instruments not provided for in groups by using measurement of speed or acceleration executed aboard the object being navigatedDead reckoning by integrating acceleration or speed, i.e. inertial navigation
G06T 7/80 - Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
G06T 19/00 - Manipulating 3D models or images for computer graphics
78.
IDENTIFICATION OF PHYSICAL PRODUCTS FOR AUGMENTED REALITY EXPERIENCES IN A MESSAGING SYSTEM
The subject technology receives image data including a representation of a physical item. The subject technology analyzes the image data to determine an object corresponding to the physical item. The subject technology extracts product metadata based on the determined object. The subject technology sends, to a server, the product metadata to determine second product metadata associated with the product metadata. The subject technology receives, from the server, the second product metadata, the second product metadata including additional information related to the physical item. The subject technology causes display, at a client device, the additional information related to the physical item based at least in part on the second product metadata.
G06F 16/583 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
G06Q 20/32 - Payment architectures, schemes or protocols characterised by the use of specific devices using wireless devices
An extended Reality (XR) system provides methodologies for displaying virtual objects in a hand-centric XR experience. The XR system provides an XR user interface of an XR system to a user. The XR system captures video frame data of a hand of the user and detects the hand of the user based on the video frame data and a hand-detecting model. The XR system generates a cropping boundary box based on the detection of the hand and the video frame data and generates cropped video frame data based on the cropping boundary box and the video frame data. The XR system generates a 3D model of a portion of the hand of the user based on the cropped video frame data and a virtual object based on the 3D model of the portion of the hand of the user and a 3D texture. The XR displays the virtual object in the XR user interface.
Systems, devices, media, and methods are presented for generating ideograms from a set of images received in an image stream. The systems and methods detect at least a portion of a face within the image and identify a set of facial landmarks within the portion of the face. The systems and methods determine one or more characteristics representing the portion of the face, in response to detecting the portion of the face. Based on the one or more characteristics and the set of facial landmarks, the systems and methods generate a representation of a face. The systems and methods position one or more graphical elements proximate to the graphical model of the face and generate an ideogram from the graphical model and the one or more graphical elements.
A generative artificial intelligence (AI) system includes a generative AI model configured to generate outputs based on a training data set. The AI system additionally includes a secure data vault system. The secure data vault system additionally includes a sandbox system storing the generative AI model and operatively coupled to the generative AI model to send inputs to generate the outputs from the generative AI model, wherein the sandbox system comprises an execution environment configured to restrict execution of the generative AI model to a predefined memory address range. The secure data vault system further includes a secure network service communicatively coupled to the sandbox system and configured to authenticate a connection to an external system and to download from the external system an update package for the generative AI model when the connection is authenticated.
G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
Systems and methods for receiving a set of media content items including a geohash defining a captured time and a captured location of the media content item, identifying a first subset of media content items from the set of media content items comprising a geohash that equals a precision level threshold, and identifying a second subset of media content items from the set of media content items that include a geohash that exceeds the precision level threshold. The system also includes generating a timeline media content item collection including the second subset of media content items each including a geohash that exceeds the precisions level threshold, and causing display of a media content collection interface, the media content collection interface including the timeline media content item collection.
G06F 16/48 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
G06F 16/487 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
A system captures via one or more sensors of a computing device, data of an environment observed by the one or more sensors at a first timeslot, and stores the data in a data store as a first portion of a timelapse memory experience. The system also captures, via the one or more sensors of a computing device, data of the environment observed by the one or more sensors at a second timeslot, and stores the data in a data store as a second portion of the timelapse memory experience. The system additionally associates the timelapse memory experience with a memory experience trigger, wherein the memory experience trigger can initiate a presentation of the timelapse memory experience.
Described is a system for performing a set of machine learning model training operations that include: accessing media content items associated with interaction functions initiated by users of an interaction system, generating training data including labels for the media content items, extracting features from a media content item of the media content items, identifying additional media content items to include in the training data based on the extracted features from the media content item, processing the training data using a machine learning model to generate a media content item output; and updating one or more parameters of the machine learning model based on the media content item output. The system checks whether retraining criteria has been met, and repeats the set of machine learning model training operations to retrain the machine learning model.
H04N 21/25 - Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication or learning user preferences for recommending movies
G06V 10/774 - Generating sets of training patternsBootstrap methods, e.g. bagging or boosting
G06V 20/40 - ScenesScene-specific elements in video content
H04N 21/234 - Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
85.
AUTOMATIC TECHNIQUES FOR CONSTRUCTING AN EVOLVING INTEREST TAXONOMY FROM USER-GENERATED CONTENT
Techniques for creating an interest graph include obtaining content items from multiple content sources and applying tailored (e.g., source-specific) preprocessing to the content items based on their respective content source. Text is extracted and salient keywords and key phrases are identified using unsupervised machine learning models. The keywords and key phrases become nodes in an interest graph, each node comprising an embedding of a keyword or key phrase in a common embedding space, with edges representing semantic similarity based on embeddings or co-engagement patterns. The graph provides an expansive, granular, and dynamic taxonomy easily adaptable to emerging interests. The interest graph overcomes limitations of conventional taxonomies that lack depth, fail to capture niche interests, and cannot adapt to reflect evolving user preferences. The described techniques construct a rich interest graph from diverse content for improved content understanding.
Eyewear including a sensor integrated into frame of eyewear. In one example, the sensor comprises a strain gauge, such as a metallic foil gauge, that is configured to sense and measure distortion of the frame when worn by a user and under different force profiles, by measuring a strain in the frame when bent. The measured strain by strain gauge is sensed by a processor, and the processor performs dynamic calibration of image processing based on the measured strain. The distortion measured by the strain gauge is used by the processor to correct calibration of the cameras, and the displays.
G01B 7/16 - Measuring arrangements characterised by the use of electric or magnetic techniques for measuring the deformation in a solid, e.g. by resistance strain gauge
G01B 11/22 - Measuring arrangements characterised by the use of optical techniques for measuring depth
H04N 13/00 - Stereoscopic video systemsMulti-view video systemsDetails thereof
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
A photo filter (e.g., artistic/stylized painting) light field effect system includes an eyewear device having a frame, a temple connected to a lateral side of the frame, and a depth-capturing camera. Execution of programming by a processor configures the stylized image painting effect system to apply a photo filter selection to: (i) a left raw image or a left processed image to create a left photo filter image, and (ii) a right raw image or a right processed image to create a right photo filter image. The stylized image painting effect system generates a photo filter stylized painting effect image with an appearance of a spatial rotation or movement, by blending together the left photo filter image and the right photo filter image based on a left image disparity map and a right image disparity map.
Methods and systems are disclosed for using machine learning models to perform smoothing in latent space using optical flow information. The methods and systems access a first frame of a video depicting an object and a second frame of the video, the second frame corresponding to a later time period in the video than the first frame. The methods and systems generate optical flow information based on the first frame and the second frame, the optical flow information describing movement of the object from the first frame to the second frame. The methods and systems smooth a latent space generated by one or more neural network encoders of a machine learning model using the optical flow information and process the smoothed latent space by one or more neural network decoders to generate a result of the machine learning model.
G06V 10/40 - Extraction of image or video features
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/62 - Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extractionPattern tracking
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G06V 20/40 - ScenesScene-specific elements in video content
G06V 40/20 - Movements or behaviour, e.g. gesture recognition
G06V 40/10 - Human or animal bodies, e.g. vehicle occupants or pedestriansBody parts, e.g. hands
Methods and systems are disclosed for training a user-specific machine learning (ML) model to detect inner speech. The system accesses the ML model trained to detect inner speech based on a general population dataset. The system collects, by an electromyograph (EMG) communication device, a set of EMG signals generated based on an individual user of the EMG communication device. The system updates parameters of the ML model based on the set of EMG signals associated with the individual user. The system detects inner speech of the individual user by applying the ML model with the updated parameters to a new set of EMG signals received from the EMG communication device.
Methods and systems are disclosed for using machine learning models to perform smoothing in latent space using optical flow information. The methods and systems access a first frame of a video depicting an object and a second frame of the video, the second frame corresponding to a later time period in the video than the first frame. The methods and systems generate optical flow information based on the first frame and the second frame, the optical flow information describing movement of the object from the first frame to the second frame. The methods and systems smooth a latent space generated by one or more neural network encoders of a machine learning model using the optical flow information and process the smoothed latent space by one or more neural network decoders to generate a result of the machine learning model.
A method for increasing a dynamic range of a dual-pixel image sensor is described. The method includes detecting an intensity level of a full pixel from a plurality of pixels of an optical sensor, one or more full pixels of the plurality of pixels includes at least two sub-pixels, detecting an intensity level of one or more sub-pixels, detecting that the intensity level of the full pixel of the optical sensor has reached a saturation level of the full pixel, and in response to detecting that the intensity level of the full pixel of the optical sensor has reached the saturation level of the full pixel, computing an extrapolated intensity level of the full pixel based on the intensity level of the one or more sub-pixels.
A system receives an offer code from a device associated with a user account. The offer code is obtained from an optical code featuring a custom graphic. The system identifies an offer from a merchant in a database based on the offer code, with the database specifying one or more parameters for the offer. An association between the offer and the user account is stored. The system receives a purchase code from the device and determines that the purchase code is associated with the merchant's offer. Upon authorizing a transaction linked to the purchase code, the system applies an offer benefit to the transaction, based on the one or more parameters, and presents a notification of the applied offer benefit at the device.
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
H04L 51/52 - User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail for supporting social networking services
An extended Reality (XR) system provides methodologies for scaling a virtual object in an XR user interface of the XR system. The methodologies include providing to a user an XR user interface of an XR system, where the XR user interface includes a virtual object displayed to the user. The XR system determines a pinch location of a pinch hand pose being made by the user and scales the virtual object based on the pinch location and a virtual object center point of the virtual object. The XR system redisplays the scaled virtual object to the user in the XR user interface.
An extended Reality (XR) system that provides services for determining 3D data of physical objects in a real-world scene. The XR system receives a request from an application to initiate a spatial scan of a real-world scene. In response, the XR system captures video frame data of the real-world scene and captures a pose of the XR system. The XR system determines a physical object in the real-world scene and determines a 2D position of the physical object, using the video frame data. The XR system determines a depth of the physical object using the 2D position and determines a 3D position of the physical object in the real-world scene using the 2D position of the physical object, the depth of the physical object, and the pose of the XR system. The XR system communicates the 3D position data to the application.
42 - Scientific, technological and industrial services, research and design
Goods & Services
Providing online non-downloadable computer software for
creating augmented reality experiences; providing online
non-downloadable computer software for creating video games;
providing online non-downloadable computer software for
modifying images, video, audio, and audio-visual content
with digital filters and augmented reality effects, namely,
text, graphics, animations, and links; providing online
non-downloadable computer software, namely, software for
integrating electronic data with real world environments for
the purpose of experiencing, viewing, capturing, recording,
manipulating, and editing augmented real-time views, images,
videos, audio, and sensory content; providing online
non-downloadable computer software, namely, augmented
reality software for object segmentation, object tracking,
tracking of dwellings and structures, and hand tracking;
providing online non-downloadable augmented reality software
for modifying images, video, audio and audio-visual content
that are used in the fields of entertainment, utilities, and
communication; providing online non-downloadable software
for creating augmented reality software.
A pose tracking system is provided. The pose tracking system includes an EMF tracking system having a user-worn head-mounted EMF source and one or more user-worn EMF tracking sensors attached to the wrists of the user. The EMF source is associated with a VIO tracking system such as AR glasses or the like. The pose tracking system determines a pose of the user's head and a ground plane using the VIO tracking system and a pose of the user's hands using the EMF tracking system to determine a full-body pose for the user. Metal interference with the EMF tracking system is minimized using an IMU mounted with the EMF tracking sensors. Long term drift in the IMU and the VIO tracking system are minimized using the EMF tracking system.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
A method for generating a virtual selfie stick image is described. In one aspect, the method includes generating, at a device, an original self-portrait image with an optical sensor of the device, the optical sensor directed at a face of a user of the device, the device being held at an arm length from the face of the user, displaying, on a display of the device, an instruction guiding the user to move the device at the arm length about the face of the user within a limited range at a plurality of poses, accessing, at the device, image data generated by the optical sensor at the plurality of poses, and generating a virtual selfie stick self-portrait image based on the original self-portrait image and the image data.
A method is disclosed comprising accessing an image; identifying a virtual object corresponding to a physical object depicted in the image; and determining shading parameters for the virtual object based on a machine learning model. The model is trained by generating a synthetic face image using a first renderer; predicting lighting parameters from the synthetic face image with a neural network; generating a predicted sphere image using a second renderer based on the predicted lighting parameters; generating a synthetic sphere image using a third renderer; comparing the predicted sphere image with the synthetic sphere image; and training the neural network based on the comparison. The method further comprises generating a shaded virtual object by applying the shading parameters to the virtual object and displaying the shaded virtual object as a layer over the image.
An ergonomics evaluation system is described. In one aspect, a method includes identifying a placement location of a user interface element of an augmented reality application, identifying simulated user interactions of the augmented reality application based on the placement location of the user interface element, applying a computer vision algorithm to the simulated user interactions, identifying a joint angle of a user based on an output of the computer vision algorithm, and identifying an ergonomic risk level of the joint angle.
A method for aligning coordinate systems between devices is described. The method comprises accessing pose data from a first device that defines a first coordinate system and receiving pose data from a second device that defines a second coordinate system. The method further includes identifying a coordinate transformation that relates the first device, a hand joint of a user holding the second device, and the second device, and then aligning the first coordinate system with the second based on the pose data and the identified transformation.
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06T 7/70 - Determining position or orientation of objects or cameras
G06T 7/80 - Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
