A computer-implemented method produces latent representations of text prompts created for use with a text-to-image diffusion model. Training images are generated by providing the latent representations to a first artificial neural network implementing a denoising process of the text-to-image diffusion model. A machine-learned modality inversion module is trained. The training includes performing training iterations for training data pairs, each training data pair being comprised of one of the training images and one of the text prompts. Each training iteration for each training data pair includes: providing the one of the training images of the training data pair to a pre-trained classifier configured to generate alternate conditioning information based upon the one of the training images, converting, by the machine-learned modality inversion module, the alternate conditioning information into an alternate latent representation, and updating parameters of the machine-learned modality inversion module based upon differences between the alternate latent representation and one of the latent representations of the one of the text prompts.
A computer-implemented method for image generation includes receiving conditioning data derived from imagery within input data frames. Auxiliary audio data derived from words or sounds vocalized by a subject within the input data frames is received. The audio data is included within audio content associated the input data frames. The conditioning data and the auxiliary audio data is provided to a composite artificial neural network where the composite artificial neural network is configured to perform a controlled diffusion process. The composite artificial neural network includes a neural network implementing a diffusion model in combination with a control neural network. The composite artificial neural network generates reconstructed versions of the input data frames based upon the conditioning data and the auxiliary audio data.
A method includes receiving input frames of video information. An uplink channel receives a requirements indication from a mobile device configured to implement a diffusion model. Based upon the requirements indication, a current video coding modality is selected from among a first video coding modality and a second video coding modality where the first video coding modality utilizes diffusion, and the second video coding modality does not utilize diffusion. Video coding data is generated by processing the input frames of video information using the current video coding modality. The video coding data is sent to the mobile device.
H04N 19/70 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
4.
NOVEL VIEW SYNTHESIS FOR FACILITATING EYE-TO-EYE CONTACT DURING VIDEOCONFERENCING
A computer-implemented method includes receiving at a first computing device a first machine-learned latent space model of a first scene located proximate a second computing device where the first scene includes a first face. The first machine-learned latent space model of the first scene is decoded to produce first generated imagery corresponding to the first scene from a first viewpoint defined by a first virtual camera. The first virtual camera is positioned in alignment with a display screen of the second computing device.
A pioneering parallel diffusion technique individually represents and diffuses each bit or groups of bits. The approach addresses inefficiencies observed in traditional diffusion processes. The approach may reduce the number of iterations required for denoising, thereby decreasing denoising latency and improving overall processing speed. These advantages are especially crucial in the realm of codec applications where real-time processing and resource efficiency are paramount.
A computer-implemented method includes generating a set of weights for a diffusion model. The generating includes reducing fidelity of training frames of training image data to create frames of reduced-fidelity training image data, encoding the frames of reduced-fidelity training image data to create frames of compressed reduced-fidelity training image data, and training a first artificial neural network using the frames of compressed reduced-fidelity training image data where values of the weights are adjusted during the training. The values of the weights are sent to a computing device configured to use the values of the weights to establish a second artificial neural network configured to substantially replicate the first artificial neural network.
H04N 19/42 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
H04N 19/136 - Incoming video signal characteristics or properties
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/463 - Embedding additional information in the video signal during the compression process by compressing encoding parameters before transmission
7.
SPOTLIGHT TRAINING OF LATENT MODELS USED IN VIDEO COMMUNICATION
A computer-implemented method includes receiving training data with training images of a scene and associated camera extrinsics corresponding to three-dimensional (3D) camera locations and camera directions from which the training images are captured. Using the training data, a neural network is trained to represent a latent model of the scene in a latent space where the neural network is configured to synthesize scene images corresponding to novel views of the scene from queried 3D viewpoints and viewing angles. View spotlight information is received. The training of the neural network is prioritized based upon the view spotlight information.
H04N 13/117 - Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06T 7/80 - Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
G06V 10/25 - Determination of region of interest [ROI] or a volume of interest [VOI]
A computer-implemented method for generating image sequences includes receiving, at a computing device, values of a set of weights for a diffusion model. The weights are generated by training a first artificial neural network using training frames of training image data in combination with a first set of data derived from the frames of training image data. The values of the set of weights are adjusted during the training. A second artificial neural network present on the computing uses the values of the set of weights and receives a second set of data derived from frames of image data containing at least some scene information present in the training frames of training image data. The second set of data is provided to the second artificial neural network to implement the diffusion model. Images corresponding to the frames of image data are then generated by the second artificial neural network.
A computer implemented method includes receiving keyframe images of a scene captured at an initial time and first images of the scene captured a first time following the initial time. Each of the keyframe images is associated with a corresponding three-dimensional (3D) camera location and camera direction included within a set of keyframe camera extrinsics and each of the first frame images is associated with a corresponding 3D camera location and camera direction included within a set of first frame camera extrinsics. A keyframe neural network is trained using the keyframe images and the keyframe camera extrinsics. A first frame neural network is trained using the first frame images and the first frame camera extrinsics. The first frame neural network is configured to be queried to produce a first novel view of an appearance of the scene at the first time.
A system and method for generating 3D-aware reconstructions of a static or dynamic scene by using a neural network to encode captured images of the scene into a compact spatio-temporal polynomial latent space scene model. Image frames of a scene are received. Each image frame is associated with camera extrinsics including a three-dimensional (3D) camera location and a camera direction. A neural network is trained using the image frames (e.g., video frames) and the camera extrinsics to encode the frames as models of the scene in a spatio-temporal polynomial latent space. The models of the scene are transmitted to a viewing device including a latent model decoder. The latent model decoder is configured to decode the models to generate imagery corresponding to novel 3D views of the scene.
A system and method generates 3D-aware reconstructions of a static or dynamic scene by using a neural network to encode captured images of the scene into a compact latent space scene model. The method includes receiving image frames of a scene where each image frame is associated with camera extrinsics including a three-dimensional (3D) camera location and a camera direction. A neural network is trained using the image frames (e.g., video frames) and the camera extrinsics to encode the frames as models of the scene in a latent space associated with a latent model decoder. The method further includes transmitting one or more of the models of the scene to a viewing device including the latent model decoder. The latent model decoder is configured to decode the models to generate imagery corresponding to novel 3D views of the scene.
A system and method for generating 3D-aware reconstructions of a static or dynamic scene by using a neural network to encode captured images of the scene into a compact spatio-temporal polynomial latent space scene model. Image frames of a scene are received. Each image frame is associated with camera extrinsics including a three-dimensional (3D) camera location and a camera direction. A neural network is trained using the image frames (e.g., video frames) and the camera extrinsics to encode the frames as models of the scene in a spatio-temporal polynomial latent space. The models of the scene are transmitted to a viewing device including a latent model decoder. The latent model decoder is configured to decode the models to generate imagery corresponding to novel 3D views of the scene.
A system and method generates 3D-aware reconstructions of a static or dynamic scene by using a neural network to encode captured images of the scene into a compact latent space scene model. The method includes receiving image frames of a scene where each image frame is associated with camera extrinsics including a three-dimensional (3D) camera location and a camera direction. A neural network is trained using the image frames (e.g., video frames) and the camera extrinsics to encode the frames as models of the scene in a latent space associated with a latent model decoder. The method further includes transmitting one or more of the models of the scene to a viewing device including the latent model decoder. The latent model decoder is configured to decode the models to generate imagery corresponding to novel 3D views of the scene.
A power bank and holographic projection accessory intended for use with a portable electronic device. The accessory includes a holographic projection unit and an external power bank case attachable to the holographic projection unit. The external power bank case is configured to provide power to the holographic projection unit and to provide a video signal generated by the portable electronic device to the holographic projection unit. The holographic projection unit generates volumetric projections based upon the video signal.
A power bank and holographic projection accessory intended for use with a portable electronic device. The accessory includes a holographic projection unit and an external power bank case attachable to the holographic projection unit. The external power bank case is configured to provide power to the holographic projection unit and to provide a video signal generated by the portable electronic device to the holographic projection unit. The holographic projection unit generates volumetric projections based upon the video signal.
G02B 30/60 - Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images involving reflecting prisms and mirrors only
A holographic display system including an electronic device, a camera and a holographic projection unit. The holographic projection unit is configured to generate a volumetric projection for viewing by a user in response to a rendering signal provided by a volumetric display application executing on the electronic device. The holographic projection unit includes a housing, a projector at least partially disposed within the housing and operative to display images based upon the rendering information, and a semi-reflective element being oriented to reflect light from the images in order to create the volumetric projection. The camera is oriented such that the user is within a field of view, the camera being operative to provide the image information to the volumetric display application for determination of a position of the user. The volumetric projection is adapted in response to the position of the user.
A power bank and holographic projection accessory intended for use with a portable electronic device. The accessory includes a holographic projection unit and an external power bank case attachable to the holographic projection unit. The external power bank case is configured to provide power to the holographic projection unit and to provide a video signal generated by the portable electronic device to the holographic projection unit. The holographic projection unit generates volumetric projections based upon the video signal.
A holographic display system including an electronic device, a camera and a holographic projection unit. The holographic projection unit is configured to generate a volumetric projection for viewing by a user in response to a rendering signal provided by a volumetric display application executing on the electronic device. The holographic projection unit includes a housing, a projector at least partially disposed within the housing and operative to display images based upon the rendering information, and a semi-reflective element being oriented to reflect light from the images in order to create the volumetric projection. The camera is oriented such that the user is within a field of view, the camera being operative to provide the image information to the volumetric display application for determination of a position of the user. The volumetric projection is adapted in response to the position of the user.
G03H 1/02 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto Details
G02B 27/14 - Beam splitting or combining systems operating by reflection only
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 1/16 - Constructional details or arrangements
A micro-layered structure for use in an optical projection system is configured to reflect incident light as multiple closely aligned reflections. A first semi-reflective transparent layer of the structure has a first index of refraction, a first front surface and a first rear surface. A second semi-reflective transparent layer of the structure has a second index of refraction different from the first index of refraction, a second front surface and a second rear surface. The second front surface abuts the first rear surface of the first semi-reflective transparent layer. Portions of the incident light are respectively reflected by the first front surface, first rear surface, second front surface and second rear surface as first, second, third and fourth reflections which collectively form a projection of the image perceived by a user of the optical projection system.
A micro-layered structure for use in an optical projection system is configured to reflect incident light as multiple closely aligned reflections. A first semi-reflective transparent layer of the structure has a first index of refraction, a first front surface and a first rear surface. A second semi-reflective transparent layer of the structure has a second index of refraction different from the first index of refraction, a second front surface and a second rear surface. The second front surface abuts the first rear surface of the first semi-reflective transparent layer. Portions of the incident light are respectively reflected by the first front surface, first rear surface, second front surface and second rear surface as first, second, third and fourth reflections which collectively form a projection of the image perceived by a user of the optical projection system.
A holographic display system for a portable electronic device. The display system includes a case configured to receive the portable electronic device and a projector coupled to the case by a first hinge element. The projector includes a projector screen for generating images. A reflective element is coupled to the projector by a second hinge element. The reflective element is oriented to reflect light from the images in order to create holographic images perceptible to a user of the portable electronic device. The case may include a connector for receiving, from the portable electronic device, a video signal defining the images. The holographic display system may further include a substantially transparent touch screen attached to the first hinge element.
G03H 1/00 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto
H04M 1/02 - Constructional features of telephone sets
G03H 1/02 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto Details
25.
PORTABLE TERMINAL ACCESSORY DEVICE FOR HOLOGRAPHIC PROJECTION AND USER INTERFACE
A holographic display system for a portable electronic device. The display system includes a case configured to receive the portable electronic device and a projector coupled to the case by a first hinge element. The projector includes a projector screen for generating images. A reflective element is coupled to the projector by a second hinge element. The reflective element is oriented to reflect light from the images in order to create holographic images perceptible to a user of the portable electronic device. The case may include a connector for receiving, from the portable electronic device, a video signal defining the images. The holographic display system may further include a substantially transparent touch screen attached to the first hinge element.
G03H 1/22 - Processes or apparatus for obtaining an optical image from holograms
G06F 1/16 - Constructional details or arrangements
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
26.
PORTABLE TERMINAL ACCESSORY DEVICE FOR HOLOGRAPHIC PROJECTION AND USER INTERFACE
A holographic display system for a portable electronic device. The display system includes a case configured to receive the portable electronic device and a projector coupled to the case by a first hinge element. The projector includes a projector screen for generating images. A reflective element is coupled to the projector by a second hinge element. The reflective element is oriented to reflect light from the images in order to create holographic images perceptible to a user of the portable electronic device. The case may include a connector for receiving, from the portable electronic device, a video signal defining the images. The holographic display system may further include a substantially transparent touch screen attached to the first hinge element.
G03H 1/22 - Processes or apparatus for obtaining an optical image from holograms
G06F 1/16 - Constructional details or arrangements
G06F 3/0346 - Pointing devices displaced or positioned by the userAccessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
G09G 3/20 - Control arrangements or circuits, of interest only in connection with visual indicators other than cathode-ray tubes for presentation of an assembly of a number of characters, e.g. a page, by composing the assembly by combination of individual elements arranged in a matrix
27.
Portable terminal accessory device for holographic projection and user interface
A holographic display system for a portable electronic device. The display system includes a case configured to receive the portable electronic device and a projector coupled to the case by a first hinge element. The projector includes a projector screen for generating images. A reflective element is coupled to the projector by a second hinge element. The reflective element is oriented to reflect light from the images in order to create holographic images perceptible to a user of the portable electronic device. The case may include a connector for receiving, from the portable electronic device, a video signal defining the images. The holographic display system may further include a substantially transparent touch screen attached to the first hinge element.
H04M 1/00 - Substation equipment, e.g. for use by subscribers
H04M 1/02 - Constructional features of telephone sets
G03H 1/02 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto Details
G03H 1/00 - Holographic processes or apparatus using light, infrared, or ultraviolet waves for obtaining holograms or for obtaining an image from themDetails peculiar thereto
patents
