In various examples, systems and methods for synthetic data generation, augmentation, and modification using diffusion based techniques are provided. One or more diffusion models (e.g., latent diffusion models) may be used to generate synthetic data (e.g., images) based on semantic image(s) and/or textual information. The diffusion model(s) may be trained using datasets that include corresponding ground truth images, semantic images, and textual information. Semantic image embedding(s) may be generated by encoding received semantic image(s), and text embedding(s) may be generated by encoding received textual information. The semantic image embedding(s) may be concatenated with noise to generate concatenated embedding(s). Image(s) may be generated from noise or the concatenated embedding(s) using the diffusion model(s). A denoising network of the diffusion model(s) may be conditioned using the text embedding(s) through one or more cross-attention mechanisms.
Key-value (KV) management processes can be a training-free approach to KV cache compression, addressing the challenges of memory bandwidth and capacity demands during the decode phase of source material using large language model (LLM) inference. The KV management processes can contain two consecutive stages: a KV cache eviction algorithm for coarse-grain KV cache eviction and a hybrid attention process for fine-grain dynamic KV token selection. The hybrid attention process can have three steps, (1) grouping KV pages along the sequence dimension while storing the minimum and maximum values, (2) accumulating the query data in a group dimension finding the largest index along the head dimension, and then performing an approximation attention, and (3) a sparse attention can be performed by fetching the KV pair vectors from the top K indices. KV management processes can maintain comparable accuracy to KV cache attention while lowering memory bandwidth and capacity usage.
Apparatuses, systems, and techniques to use neural networks to predict vertices to add to a surface mesh of a three-dimensional (3D) object. In at least one embodiment, one or more neural networks predict one or more verticies to add to a 3D object based, at least in part, on different vertices removed from surface meshes of different 3D objects.
In various examples, systems and methods are disclosed relating to aliasing control of program variables in storage via automatic application of artificial dependences during program compilation. In some implementations, a system can include a detector to automatically detect a pattern, based at least on a structure of data flow in a source program, indicative of sequences of dependent operations, where the sequences are independent from one another. The system can determine a storage aliasing preference for whether to allow the compiler to allocate the program variables of the respective sequences to the same processor storage locations, or to prevent the compiler from doing so. The system can assign one or more annotations to the source program indicative of one or more artificial dependences for a compiler to respect when performing program transformations prior to the allocation of program variables.
Mechanisms for the generation of randomized sequences of substantial length the generation of which involves the frequent exchange of long bit sequences between threads in a multiprocessor. The mechanisms accelerate in hardware the swapping of long sequences between threads that in conventional systems slows the generation of the sequences down and increases the power consumption needed to generate the sequences.
Apparatuses, systems, and techniques to scale tensors of one or more neural networks. In at least one embodiment, a processor includes one or more circuits to cause two or more scaling factors of two or more corresponding portions of one or more tensors to be scaled using one or more randomly generated bits.
G06F 7/58 - Random or pseudo-random number generators
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
At least one embodiment for generating virtual environments using vision-language-action models includes receiving an input text prompt, generating a first virtual environment based on the input text prompt, generating a second virtual environment by adding a plurality of scene elements to the first virtual environment, and generating a third virtual environment by adding a plurality of additional assets to the second virtual environment, the additional assets being smaller than a first scene element in the plurality of scene elements.
In various examples, perception processing architectures for semi-autonomous and autonomous systems and applications is described herein. Systems and methods described herein may use an end-to-end learned detection, tracking, and classification architecture to determine information associated with objects located within an environment. For instance, the architecture may include one or more machine learning models that process input data representing at least sensor representations (e.g., images), machine motion, and/or sensor calibrations to generate output data representing detection information, tracking information, and/or classification information associated with one or more objects As described herein, in some examples, the machine learning model(s) may include at least an encoding model, a detection/classification model, and a tracking model. Additionally, the machine learning model(s) may be trained using an end-to-end training technique such that the architecture is able to jointly perform object detection, tracking, and classification.
G06T 7/246 - Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 10/77 - Processing image or video features in feature spacesArrangements for image or video recognition or understanding using pattern recognition or machine learning using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]Blind source separation
G06V 10/774 - Generating sets of training patternsBootstrap methods, e.g. bagging or boosting
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestriansRecognition of traffic objects, e.g. traffic signs, traffic lights or roads
9.
APPLICATION PROGRAMMING INTERFACE TO INDICATE ATTRIBUTES OF GROUPS OF BLOCKS OF THREADS
Apparatuses, systems, and techniques to execute CUDA programs. In at least one embodiment, an application programming interface is performed to indicate one or more attributes of one or more groups of blocks of one or more threads.
Apparatuses, systems, and methods to neural network optimizations. In at least one embodiment, a processor comprises one or more circuits to cause a first portion of data generated by one or more instructions to be used by one or more other instructions while a second portion of the data is being transmitted based, at least in part, on a store granularity of a store instruction.
Apparatuses, systems, and techniques using neural networks to generate animations and/or models. In at least one embodiment, one or more processors comprise circuitry to obtain constraints on character attributes, generate noise representations to comprise representations of attributes conforming to the constraints, and use one or more neural networks to generate frames of video based on denoising the noise representations.
Example embodiments described herein relate to a resource scheduling system that incorporates user sentiment analysis to enhance responsiveness in computing environments. The embodiments include a lightweight model for estimating compute requirements based on user prompts and organization-specific inputs, as well as a second model for predicting response times based on hardware allocation. Additionally, it features a sentiment computation model that evaluates user engagement metrics and feedback to generate sentiment scores. A control panel allows system administrators to prioritize resource allocation by adjusting the influence of compute requirements, response times, and user sentiment.
Network switches are devices that connect multiple devices together on a computer network, using packet switching to receive, process, and forward data to the destination device. Each switch typically contains multiple ports, which are the points of connection for network cables. These ports can be in an active state, where they are ready to transmit data, or in an idle state, where they consume less power. Power consumption in datacenters has been a topic of concern due to the increasing demand for data processing and storage. One approach to reducing power consumption involves managing the power state of the switch ports. However, current power saving policies focus on making decisions for one type of traffic pattern or for a single port at a time, and therefore cannot intelligently or dynamically adapt to a multitude of network parameters affecting traffic flows. The present disclosure uses artificial intelligence to more intelligently transition ports between different modes of operation.
ƒ-distill is a framework for distributional matching distillation based on ƒ-divergence minimization. A student diffusion model (DM) can be trained by generating an output content, scoring a student distribution of data values, and then comparing the student scores to a teacher DM scoring. The scoring process can use a gradient algorithm that scores multiple regions of the generated content individually. The gradient algorithm can comprise the product of the weighting function and the score difference between the teacher and student distributions across each region of the generated content. Each region's score can be weighted separately from other regions based on data distributions, mode-seeking parameters, or other factors. ƒ-distill can encompass variational score distillation objectives while allowing less mode-seeking divergences. In some aspects, by leveraging the weighting function, ƒ-distill can down weight regions with larger score estimation errors.
A processing system includes a first parallel processing unit comprising a transmitter with a finite impulse response filter, and a second parallel processing unit comprising a receiver with a linear equalizer. A communication link couples the transmitter to the receiver. Calibration circuitry determines a pulse response at an output of the linear equalizer based on one or more symbols transmitted by the transmitter over the communication link, and configures one or more post-cursor taps of the finite impulse response filter based on the pulse response.
A multi-dielet module is disclosed. In one embodiment, the module comprises an interposer and a substrate. The interposer is connected to at least one I/O dielet and at least one non-I/O dielet using micro-bump interconnect structures. The substrate is connected to the interposer using C4 bumps on one surface of the substrate and BGA interconnect structures on another. The at least one I/O dielet is configured to allow structural/scan test data from an external source to be communicated through the substrate and interposer to test I/Os of the at least one I/O dielet. The at least one I/O dielet is configured to allow the structural/scan test data to be communicated from the at least one I/O dielet to the that at least one non-I/O dielet using a chip-to-chip (C2C) interface and the micro-bump interconnect structures of the at least one I/O dielet and the at least one non-I/O dielet.
Apparatuses, systems, and techniques to translate information of neural network portions to perform neural network portions. In at least one embodiment, one or more translation neural networks are used to translate information used between two or more portions of two or more neural networks to perform inferencing based on the translated information.
This specification describes a programmatic multicast technique enabling one thread (for example, in a cooperative group array (CGA) on a GPU) to request data on behalf of one or more other threads (for example, executing on respective processor cores of the GPU). The multicast is supported by tracking circuitry that interfaces between multicast requests received from processor cores and the available memory. The multicast is designed to reduce cache (for example, layer 2 cache) bandwidth utilization enabling strong scaling and smaller tile sizes.
In various examples, machine learning model-based customized real-time sign language translation is provided. A sign language translation framework may receive a data feed and detect when translation services are needed and a particular form of sign language is being used. The framework may translate incoming spoken language into a selected sign language to be used at the client application and generate video content to display the translated sign language content to the user interface of the client application. Sign language video data may comprise a representation of an animated avatar presented as an overlay performing signing corresponding to sign language translation data. The framework may generate an augmented video feed that modifies the appearance of a meeting participant to show the participant performing the sign language translation. A translation framework may conversely translate incoming video frames depicting users using sign language into spoken language data that may be audibly presented.
In one embodiment, a distributed computing system includes multiple nodes to be interconnected by multiple physical links to convey traffic between the nodes, each node comprising link controller logic to control transitions of a physical link of the multiple physical links among states including an active state L0 in which the traffic is allowed to be conveyed by the physical link, a power saving state L1 in which traffic is not allowed to be conveyed by the physical link and having a first exit latency to the active state L0, and a sleep state in which traffic is not allowed to be conveyed by the physical link and providing higher power saving than the power saving state L1 and having a second exit latency to the active state L0, the second exit latency being greater than the first exit latency.
Apparatuses, systems, and techniques to quantize one or more portions of one or more neural networks. In at least one embodiment, one or more processors include one or more circuits which use one or more randomly selected scaling factors to quantize one or more portions of one or more neural networks.
G06F 5/01 - Methods or arrangements for data conversion without changing the order or content of the data handled for shifting, e.g. justifying, scaling, normalising
22.
EVALUATING SAFEGUARD MODELS FOR MODERATION OF LANGUAGE APPLICATIONS
Disclosed are apparatuses, systems, and techniques for accurate and unsupervised assessments of safety of AI operations. The techniques include processing, using an ensemble of safeguard models (SGMs), a first input to generate, by a respective SGM of the ensemble: an individual safety assessment of the first input, and a distribution, associated with the individual safety assessment, over tokens of the respective SGM. The techniques further include obtaining randomness values characterizing a degree of randomness of the generated distributions, updating, using the plurality of randomness values, a plurality of weights associated with the SGMs of the ensemble, generating, using the plurality of weights, an ensemble assessment of the first input or a second input.
Approaches presented herein provide for the automated detection of physical state information for a set of components, such as those present in a datacenter. The physical state information can be obtained by using one or more sensors (e.g., cameras) to capture visual or other physical data for one or more components. The captured sensor data can be analyzed to determine physical state aspects for various components. This observed state can be compared, using an artificial intelligence (AI) model trained on physical language, to a typical set of state data to attempt to identify any anomalies. If any anomalies are identified, an AI actor can determine whether any of those anomalies are related to a current or potential problem, and if so, can generate some type of notification or alarm to cause the problem to be investigated. In at least one embodiment, a robotic assembly can be used that can move between various components and use one or more sensors to capture at least a portion of the appropriate physical state data.
In various examples, per-pixel statistics from light sampling events—such as Next Event Estimation (NEE) sampling events—may be obtained and used to guide future light sampling during the rendering of subsequent frames or images. For instance, the systems and methods of the present disclosure may use weighted reservoir sampling to stochastically store (e.g., in image space) light sampling statistics indicative of light sources that contributed the most to the radiance of each pixel of a first rendered image. Using these statistics, one or more sampling distributions may be generated for future rendering passes of subsequent images. For instance, a global sampling distribution specific to the whole frame and one or more local sampling distributions specific to different groups of pixels (e.g., tiles) within the frame may be generated and used to guide sampling events for rendering a second image.
In various examples, systems and methods are disclosed relating to approximating motion vectors for refractive surfaces. One or more circuits can identify an object and a refractive surface of a simulated scene. The one or more circuits can generate, based at least on a ray of a ray tracing process, a transformation data structure for the object relative to the refractive surface. The one or more circuits can determine a position in the simulated scene depicted by an image representing the object appearing behind the refractive surface based at least on the transformation data structure. The one or more circuits can render a frame of the simulated scene according to the position of the image representing the object.
Apparatuses, systems, and techniques to use neural networks to predict vertices to add to a surface mesh of a three-dimensional (3D) object. In at least one embodiment, one or more neural networks predict one or more vertices to add to a 3D object based, at least in part, on different vertices removed from surface meshes of different 3D objects.
In at least one embodiment, a reinforcement-learning-based searching approach is used to produce a training configuration for a machine-learning model. In at least one embodiment, 3D medical image segmentation is performed using learned image preprocessing parameters.
Apparatuses, systems, and techniques to perform inferencing with neural networks. In at least one embodiment, portions of one or more neural networks are selected for use in inferencing based, at least in part, on information to be inferenced by the one or more neural networks.
Processors, systems and methods are described that cause information detected by one or more quantum sensors to be processed in parallel in order to identify one or more physical properties indicated by the information.
Apparatuses, systems, and techniques to share portions of storage across operating system instances. In at least one embodiment, one or more APIs are performed to cause a portion of storage used by a first operation system instance to be shared with a second operation system instance using a handle identifying the portion of storage.
09 - Scientific and electric apparatus and instruments
42 - Scientific, technological and industrial services, research and design
Goods & Services
Computer software; downloadable software development kits (SDKs); downloadable application programming interface (API); Downloadable computer software for vehicles in the field of self-driving and autonomous land vehicles and autonomous land vehicle component operation, control, maintenance, management, and communication; downloadable computer software for electronic control systems for land motor vehicles; downloadable computer software for the operation, control, maintenance, and management of land vehicles, self-driving and autonomous land vehicles and land vehicle components; downloadable computer software for land vehicle navigation, travel and trip planning, and communications; downloadable computer software for collecting, tracking, analyzing, and reporting data and information in the field of self-driving and autonomous land vehicles; downloadable computer software for machine learning, deep learning, deep neural networks, natural language generation, statistical learning, supervised learning, unsupervised learning, data mining, predictive analytics, and business intelligence; downloadable computer software for data management, analytics, and pattern and activity recognition; downloadable computer software development tools and software development kits (SDKs) for developing software for self-driving and autonomous land vehicle electronic computing platforms and systems; downloadable computer software for facilitating interaction between humans and machines; downloadable computer software for object tracking, motion control, and content visualization; downloadable computer software for visual, voice, audio, motion, eye, and gesture tracking and recognition; downloadable computer software for operating sensor devices; downloadable computer software for detecting objects, user gestures, and commands; downloadable computer software enabling hardware and electronic devices to share data and communicate with each other; downloadable computer software for vehicle fine-tuning, development, diagnostics, calibration, and configuration of electronic systems; downloadable computer graphics software for creating 3D content, 3D product renderings, photorealistic renderings, and models; downloadable computer software for integrating electronic data with real world environments for capturing, processing, and visualizing modeling and placement of goods of others and visual image markers; downloadable computer software for use as an application programming interface (API) for building software applications; downloadable computer software for use as an application programming interface (API) for developing and creating virtual reality and augmented reality experiences; downloadable computer software for developing and creating virtual reality and augmented reality experiences; Downloadable computer software for simulating autonomous vehicle driving scenarios, including urban, highway, and parking environments, for testing and validating advanced driver assistance systems and autonomous driving systems; Downloadable computer software for generating synthetic driving data and digital twin environments from real world driving data for use in training, testing, and validating artificial intelligence and machine learning models for vehicle perception, planning, and control; Downloadable computer programs for creating, configuring, and running virtual test drives and virtual proving grounds with physically accurate rendering of cameras, LiDARs, radars, and other vehicle sensors; Downloadable simulation software development tools and software development kits (SDKs) for building custom 3D simulation applications, plugins, and microservices for use with autonomous vehicle simulation platforms; Downloadable computer software for creating, editing, and managing 3D digital twins of vehicles, vehicle interiors, road networks, and driving environments for visualization, design, and validation of vehicle hardware and software systems Providing temporary use of non-downloadable computer software; providing online software platforms; software as a service (SaaS); design and development of computer software; application service provider featuring application programming interface (API); Artificial intelligence as a service (AIAAS); Software as a Service (SaaS) featuring software for vehicles in the field of self-driving and autonomous land vehicle and self-driving and autonomous land vehicle component operation, control, maintenance, management and communication; software as a Service (SaaS) featuring software for vehicles in the field of electronic control systems for land motor vehicles; providing on-line nondownloadable software for the operation, control, maintenance, and management of land vehicles, self-driving and autonomous land vehicles and land vehicle components, for land vehicle navigation, for travel and trip planning, for communications and for collecting, tracking, analyzing, and reporting data and information in the field of self-driving and autonomous land vehicles; providing on-line nondownloadable software for machine learning, deep learning, natural language generation, statistical learning, supervised learning, un-supervised learning, data mining, predictive analytics and business intelligence; providing on-line nondownloadable software for data management, analytics and pattern and activity recognition; software as a Service (SaaS) services featuring software for machine learning, deep learning, and deep neural networks; design and development of computer software in the fields of machine learning, deep learning, and deep neural networks; design and development of computer software in the fields of self-driving and autonomous land vehicle electronic computing platforms and systems; software as a service (SaaS) featuring computer software for others to use for the development of software for self-driving and autonomous land vehicle electronic computing platforms and systems; providing on-line nondownloadable software for facilitating interaction between humans and machines; providing on-line nondownloadable software for object tracking, motion control and content visualization; providing on-line nondownloadable software for use in visual, voice, audio, motion, eye and gesture tracking and recognition; providing online nondownloadable software for operating sensor devices; providing on-line nondownloadable software for detecting objects, user gestures and commands; providing temporary use of nondownloadable on-line software enabling hardware and electronic devices to share data and communicate with each other; research and design in the field of technology for vehicle technology and testing; design and development of computer software for fine-tuning, development, diagnostics for vehicles, calibration, configuration and diagnostics of electronic systems; graphics design services, namely, creating 3D content, 3D product renderings, photorealistic renderings and models; providing on-line nondownloadable software for integrating electronic data with real world environments for the purpose of capturing, processing, and visualizing modeling and placement of goods of others and visual image markers; application service provider featuring application programming interface (API) for use in building software applications; Application service provider featuring application programming interface (API) for computer software for developing and creating virtual reality and augmented reality experiences; Software as a service (SaaS) featuring software for simulating autonomous vehicle driving scenarios, including urban, highway, and parking environments, for testing and validating advanced driver assistance systems and autonomous driving systems; Software as a service (SaaS) featuring software for generating synthetic driving data and digital twin environments from real world driving data for use in training, testing, and validating artificial intelligence and machine learning models for vehicle perception, planning, and control; Platform as a service (PaaS) featuring cloud based simulation platforms for creating, configuring, and running virtual test drives and virtual proving grounds with physically accurate rendering of cameras, LiDARs, radars, and other vehicle sensors; Platform as a service (PaaS) featuring cloud based simulation software development tools and platforms for building custom 3D simulation applications, plugins, and microservices for use with autonomous vehicle simulation platforms; Software as a service (SaaS) featuring software for creating, editing, and managing 3D digital twins of vehicles, vehicle interiors, road networks, and driving environments for visualization, design, and validation of vehicle hardware and software systems
42 - Scientific, technological and industrial services, research and design
Goods & Services
Providing temporary use of non-downloadable computer software; providing online software platforms; software as a service (SaaS); design and development of computer software; application service provider featuring application programming interface (API); Artificial intelligence as a service (AIAAS); Software as a Service (SaaS) featuring software for vehicles in the field of self-driving and autonomous land vehicle and self-driving and autonomous land vehicle component operation, control, maintenance, management and communication; software as a Service (SaaS) featuring software for vehicles in the field of electronic control systems for land motor vehicles; providing on-line nondownloadable software for the operation, control, maintenance, and management of land vehicles, self-driving and autonomous land vehicles and land vehicle components, for land vehicle navigation, for travel and trip planning, for communications and for collecting, tracking, analyzing, and reporting data and information in the field of self-driving and autonomous land vehicles; providing on-line nondownloadable software for machine learning, deep learning, natural language generation, statistical learning, supervised learning, un-supervised learning, data mining, predictive analytics and business intelligence; providing on-line nondownloadable software for data management, analytics and pattern and activity recognition; software as a Service (SaaS) services featuring software for machine learning, deep learning, and deep neural networks; design and development of computer software in the fields of machine learning, deep learning, and deep neural networks; design and development of computer software in the fields of self-driving and autonomous land vehicle electronic computing platforms and systems; software as a service (SaaS) featuring computer software for others to use for the development of software for self-driving and autonomous land vehicle electronic computing platforms and systems; providing on-line nondownloadable software for facilitating interaction between humans and machines; providing on-line nondownloadable software for object tracking, motion control and content visualization; providing on-line nondownloadable software for use in visual, voice, audio, motion, eye and gesture tracking and recognition; providing online nondownloadable software for operating sensor devices; providing on-line nondownloadable software for detecting objects, user gestures and commands; providing temporary use of nondownloadable on-line software enabling hardware and electronic devices to share data and communicate with each other; research and design in the field of technology for vehicle technology and testing; design and development of computer software for fine-tuning, development, diagnostics for vehicles, calibration, configuration and diagnostics of electronic systems; graphics design services, namely, creating 3D content, 3D product renderings, photorealistic renderings and models; providing on-line nondownloadable software for integrating electronic data with real world environments for the purpose of capturing, processing, and visualizing modeling and placement of goods of others and visual image markers; application service provider featuring application programming interface (API) for use in building software applications; Application service provider featuring application programming interface (API) for computer software for developing and creating virtual reality and augmented reality experiences; Software as a service (SaaS) featuring software for simulating autonomous vehicle driving scenarios, including urban, highway, and parking environments, for testing and validating advanced driver assistance systems and autonomous driving systems; Software as a service (SaaS) featuring software for generating synthetic driving data and digital twin environments from real world driving data for use in training, testing, and validating artificial intelligence and machine learning models for vehicle perception, planning, and control; Platform as a service (PaaS) featuring cloud based simulation platforms for creating, configuring, and running virtual test drives and virtual proving grounds with physically accurate rendering of cameras, LiDARs, radars, and other vehicle sensors; Platform as a service (PaaS) featuring cloud based simulation software development tools and platforms for building custom 3D simulation applications, plugins, and microservices for use with autonomous vehicle simulation platforms; Software as a service (SaaS) featuring software for creating, editing, and managing 3D digital twins of vehicles, vehicle interiors, road networks, and driving environments for visualization, design, and validation of vehicle hardware and software systems
Various examples, systems, and methods are disclosed relating to a body generation pipeline. A first computing system can generate, according to a plurality of characteristics of a body of a subject, an initial model of the body. The first computing system further can determine a plurality of measurements of a plurality of structures of the initial model. The first computing system further can determine, by at least one neural network, based at least on the plurality of measurements, a plurality of modifications to the initial model, the at least one neural network updated according to training data including a featurized representation of example body shapes and measurements of samples of the featurized representation. The first computing system further can update the initial model according to the plurality of modifications.
Some embodiments apply occupant slot-dependent sensor selection, sensor fusion, and/or detection logic. Detection pipeline(s) and/or fusion logic may be tailored to individual occupant slots (or groups of occupant slots), and the tailored detection pipelines and/or fusion logic may be applied to detect and generate a unified representation of whether one or more conditions are present in a given occupant slot. Taking child presence detection as an example, size-based age estimation may be use for some occupant slots (e.g., those in a first row of vehicle seating), but not others (e.g., vehicle footwells). Different logic may be applied for occupant slots in a first row of vehicle seating, a second row of vehicle seating, vehicle footwells, and/or other occupant slots. As such, the way in which sensor data and/or corresponding detections is combined may be tailored to a particular occupant slot.
B60W 50/00 - Details of control systems for road vehicle drive control not related to the control of a particular sub-unit
G01S 7/41 - Details of systems according to groups , , of systems according to group using analysis of echo signal for target characterisationTarget signatureTarget cross-section
G01S 13/04 - Systems determining presence of a target
36.
CONFIGURABLE SENSOR FUSION FOR AUTONOMOUS AND SEMI-AUTONOMOUS MACHINES AND APPLICATIONS
Some embodiments apply occupant slot-dependent sensor selection, sensor fusion, and/or detection logic. In some embodiments, a sensor fusion algorithm in a base control stack (e.g., a drive stack) of an ego-machine compute platform (e.g., a vehicle compute platform) may use configurable weights, thresholds, flags, values, and/or other parameters to customize whether and how data from multiple sensors or corresponding detection pipelines (e.g., raw sensor data, processed sensor data, computer vision or deep learning perception data) is combined. Various types of tasks may be implemented using configurable sensor fusion, such as hands-on-wheel detection, body size detection, abnormal pose detection, heart rate monitoring, liveness detection, and child presence detection, to name a few examples. As such, a sensor fusion algorithm in a base control (e.g., drive) stack may be configured (e.g., by an original equipment manufacturer) to customize a (e.g., slot-dependent) sensor selection, sensor fusion, and/or detection task.
In various examples, instanced mesh tiles representative of three-dimensional (3D) terrain in a simulation environment may be modified based on locations of features (e.g., unique or high-resolution features) to be rendered in the simulation environment as part of the 3D terrain. For instance, the systems and methods of the present disclosure may identify polygons of the mesh tiles that are located within a threshold distance of the features. In some instances, the threshold distance may vary based on a level of detail associated with the mesh tiles. For instance, the threshold distance may be shorter for mesh tiles or polygons having higher levels of detail, and longer for mesh tiles or polygons having lower levels of details. The systems may cause the identified polygons to be hidden from the mesh tiles at least during a rendering of the 3D terrain by altering parameters associated with the mesh tiles.
In various examples, unique meshes may be generated and used to replace various portions of instanced mesh tiles representative of three-dimensional (3D) terrain in a simulation environment. For instance, portions (e.g., polygons) of the instanced mesh tiles may be hidden during rendering based on the portions being located within a threshold proximity of features to be rendered in the simulation environment. The systems and methods of the present disclosure may determine a geometry of the hidden portions and generate unique meshes having the same or similar geometry for replacing the hidden portions and defining attributes or structure of the features. The systems may use the unique meshes along with the visible (e.g., non-hidden) portions of the mesh tiles to render the 3D terrain and ensure seamless transitions between the 3D terrain and the features, without having to create additional, mesh tile variants.
Systems and methods are directed to a multi-layered tubular that can detect leaks. The multi-layered tubular can include sensing layers that generate an electrical signal in response to encountering a leak. The multi-layered tubular sends an electrical signal to a controller which can take responsive action. The controller may determine the location of the leak based on the electrical signal.
G01M 3/18 - Investigating fluid tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using electric detection means for pipes, cables, or tubesInvestigating fluid tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using electric detection means for pipe joints or sealsInvestigating fluid tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using electric detection means for valves
F16L 11/04 - Hoses, i.e. flexible pipes made of rubber or flexible plastics
G01M 3/16 - Investigating fluid tightness of structures by using fluid or vacuum by detecting the presence of fluid at the leakage point using electric detection means
40.
HYBRID IMITATION LEARNING FOR NEURAL MOTION CONTROL
In various examples, a technique for hybrid imitation learning includes generating, via execution of a first machine learning model, a first plurality of actions based on a first plurality of states associated with a virtual character and computing a first set of rewards based on the first plurality of actions and a reference motion for the virtual character. The technique also includes generating, via execution of the first machine learning model, a second plurality of actions based on a second plurality of states associated with the virtual character and a training target goal and computing a second set of rewards based on discriminator output generated by a second machine learning model from the second plurality of actions. The technique further includes updating one or more parameters of the first machine learning model based on the first and second sets of rewards to produce a trained machine learning model.
In various examples, supervised learning and reinforcement learning may each run at scale within corresponding clusters of compute nodes, the compute nodes in each cluster may use specialized hardware tailored to a corresponding task, and the tasks running the different clusters may collectively operate in a loop or circular workflow using the output of one task as the input to the next. For example, parallel supervised learning jobs may continuously apply the latest set of skills or experiences learned during reinforcement learning to generate an updated model, and parallel reinforcement learning jobs may continuously use the latest version of the model to learn new skills or experiences. The present techniques may be used to train and/or test humanoid robots, physical AI, or other AI systems and applications.
Inter-process communication methods that divide a total payload of a first memory command among a plurality of packets and that encode a last-transmitted packet of the plurality of packets to include a metadata code in the total payload. The memory command may be communicated between processes executing a collective operation. Payloads of the packets are written to memory in a sequential address order, such that the metadata code is written to a particular address, and the second process generates an acknowledgment of the first memory command to the first process on condition that reading the particular address returns the metadata code.
H04L 49/103 - Packet switching elements characterised by the switching fabric construction using a shared central bufferPacket switching elements characterised by the switching fabric construction using a shared memory
H04L 49/901 - Buffering arrangements using storage descriptor, e.g. read or write pointers
43.
GENERATIVE AI-BASED AGENTIC ARCHITECTURE FOR PREDICTIVE MAINTENANCE SYSTEMS AND APPLICATIONS
In various examples, a generative AI-based agentic framework may be used to answer complex queries in predictive maintenance, enabling the use of natural language queries to extract information about machines and systems. For instance, the systems and methods of the present disclosure may route an incoming query to a specific language model based on the complexity of the query, and the specific language model may be used to execute a Reasoning and Acting (ReACT) framework to generate a response to the query. In some examples, the language model may decompose the query into different parts and assign these different parts to different AI-based agents, which may use their own tools, models, algorithms, etc., to address the different parts of the decomposed query. The language model may then form the response to the query using information obtained from the agents.
Apparatuses, systems, and techniques to use user-editable configuration inforamation to use one or more computing resources to perform one or more threads. In at least one embodiment, one or more user-editable configuration files are used to cause one or more computing resources to be allocated to perform one or more software threads.
One embodiment of a method for training a machine learning model to predict motion includes estimating, using an untrained machine learning model, a first motion based on at least one first condition; generating, using the untrained machine learning model, a second motion based on at least one second condition; computing a loss based on the first motion, the second motion, and one or more ground truth motions; and updating one or more parameters of the untrained machine learning model based on the loss to generate a trained machine learning model.
One embodiment of a method for controlling a character includes receiving one or more conditions, based on the one or more conditions, generating or estimating a motion using a trained machine learning model, wherein the trained machine learning model is configured to estimate the motion via one denoising diffusion step or generate the motion via a plurality of denoising diffusion steps, and causing the character to move based on the motion.
Systems and methods for fast light field rendering from a three-dimensional (3D) representation of a scene. In at least one embodiment, fast light field rendering exploits cached color values of a plurality of color planes corresponding to a reference view and cached transmittance values of a plurality of transmittance planes corresponding to the reference view to composite a light field quilt via a single sweep through a plurality of sampling planes/volume chunks, thereby enhancing computational efficiency during rendering.
Apparatuses, systems, and techniques to cause a plurality of queues to be created and operations to be performed using the plurality of queues. In at least one embodiment, a processor comprises circuitry to cause a plurality of queues to be created between a first set of threads and a second set of threads, and cause the second set of threads to begin performing operations on a portion of data received from the first set of threads through a first queue of the plurality of queues, while one or more other portions of the data are being transmitted through one or more other queues of the plurality of queues.
Apparatuses, systems, and techniques to report predicted channel state information (CSI). In at least one embodiment, a system includes one or more circuits to compare a predicted channel state information (CSI) to a measured CSI and to cause the predicted to CSI to more closely match the measured CSI.
H04B 7/06 - Diversity systemsMulti-antenna systems, i.e. transmission or reception using multiple antennas using two or more spaced independent antennas at the transmitting station
Apparatuses, systems, and techniques to determine head poses of users and provide audio for the users. In at least one embodiment, a head pose is determined based, at least in part, on camera frame information, and an audio signal is generated, based at least in part, on the determined head pose.
Apparatuses, systems, and techniques using neural networks to generate animations and/or models. In at least one embodiment, one or more processors comprise circuitry to obtain constraints on character attributes, generate noise representations to comprise representations of attributes conforming to the constraints, and use one or more neural networks to generate frames of video based on denoising the noise representations.
Various embodiments include techniques for performing parallel edge decimation on a high resolution mesh by collapsing multiple edges in parallel by blocking only the neighbor edges of the edges selected as collapse candidates. Effectively, the disclosed techniques dynamically partition the mesh into small partitions around the collapse candidates. In this manner, the techniques identify all the edges that may be independently collapsed in a single, now parallel, iteration. Edge decimation may be performed so that certain computational geometry techniques can be efficiently applied to a simpler mesh. In so doing, the disclosed techniques preserve the history of how the edge decimation process displaces the vertices of the original mesh to generate the simplified mesh. As a result, the results of the computational geometry techniques as applied to the simplified mesh can be propagated back to the original mesh.
Apparatuses, systems, and techniques to use user-editable configuration inforamation to use one or more computing resources to perform one or more threads. In at least one embodiment, one or more user-editable configuration files are used to cause one or more computing resources to be allocated to perform one or more software threads.
Embodiments of the present disclosure relate to a system and method used to transfer data via Ethernet. The system may include memory for storing data that may be received via Ethernet packets. In particular, the Ethernet packets may include a payload that may include one or more segments and a header. The header may include a sequence number field indicating a respective sequence number that corresponds to the respective segment, and a byte offset field that may indicate a respective byte offset that may be applied to the segment. Further, the system may include hardware that may be configured to perform packet analysis operations including determining whether a previously transmitted segment was lost. The system may additionally include a processing system for performing data processing operations including storing individual segments at respective memory locations based on the respective byte offsets included in the Ethernet packets.
In various examples, systems and methods are provided to perform application-level debugging of just in time (JIT) generated kernels. The kernels can be generated to be executed on parallel processing systems and/or GPUs. During generation of machine instructions of the kernels, an identifier mapping machine instructions to corresponding function calls can be stored in a debug file. During debugging of execution of the kernel, the identifier can be retrieved to allow for the function calls to be presented along with the machine instructions.
In various examples, systems and methods are disclosed relating to generating code coverage reports without instrumented builds. A system can identify a set of functions identified in a source code repository. The system can generate a respective flag for each function of the set of functions that indicates the function is to be profiled. The system can execute each function of the set of functions identified in the source code repository to generate profiling data according to the flag. The system can generate, using the profiling data, a code coverage report for the source code repository.
Some embodiments apply occupant slot-dependent sensor selection, sensor fusion, and/or detection logic for child presence detection. In some embodiments, sensor fusion logic in a base control stack (e.g., a drive stack) of an ego-machine compute platform (e.g., a vehicle compute platform) may use configurable weights, thresholds, flags, values, and/or other parameters to customize whether and how data from multiple sensors or corresponding detection pipelines (e.g., raw sensor data, processed sensor data, computer vision or deep learning perception data) is combined. For example, camera detection result(s) and RADAR detection result(s) may be combined using configurable (e.g., slot-dependent) weights to generate a fused or unified child (and/or adult) presence classification (e.g., a binary result, detection confidence). As such, a sensor fusion algorithm in a base control (e.g., drive) stack may be configured (e.g., by an original equipment manufacturer) to customize (e.g., slot-dependent) sensor selection and/or sensor fusion logic for child presence detection.
G01S 13/86 - Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
A61B 5/00 - Measuring for diagnostic purposes Identification of persons
A61B 5/05 - Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fieldsMeasuring using microwaves or radio waves
B60N 2/26 - Seats specially adapted for vehiclesArrangement or mounting of seats in vehicles for particular purposes or particular vehicles for children
G01S 13/04 - Systems determining presence of a target
G06F 30/20 - Design optimisation, verification or simulation
58.
REDUCING RESPONSE TIMES IN CONVERSATIONAL AI SYSTEMS AND APPLICATIONS
In various examples, response times (e.g., latencies) associated with conversational artificial intelligence (AI) systems may be reduced by sharing partial results between modules or components of the conversational AI systems. For example, instead of waiting for an automatic speech recognition (ASR) system to finish converting a user utterance into text data, the systems of the present disclosure may obtain candidate prefixes for the utterance from the ASR system and use a language model to predict the complete utterance based on the candidate prefixes, as well as to generate responses to the predicted utterances. As additional information is obtained (e.g., remaining portions of the utterance), the language model may update the predicted utterance and/or the response. Additionally, in some instances the systems of the present disclosure may begin forwarding the response to a Text-To-Speech (TTS) system before the language model finishes generating the response.
Various examples, systems, and methods are disclosed relating to a vision pipeline. A first computing system can capture at least one frame of an application. The first computing system further can apply the at least one frame to at least one vision language model (VLM) to cause the at least one VLM to generate at least one scene description. The first computing system further can retrieve a set of embeddings based at least on a similarity metric between the generated at least one scene description and the set of embeddings. The first computing system further can apply the set of embeddings to at least one language model (LM) to cause the at least one LM to generate at least one input command corresponding with updating a position or action of the avatar. The first computing system further can perform the at least one input command within the application.
A63F 13/52 - Controlling the output signals based on the game progress involving aspects of the displayed game scene
A63F 13/56 - Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
60.
SYNCHRONIZING SERIAL DRIVERS USING VIRTUAL SERIAL REGISTERS
In various examples, systems and methods are disclosed relating to synchronizing serial drivers using virtual serial registers. A system can initialize a serial driver comprising a virtual register and transmit, using a first application, a first serial communication via the serial driver to configure an aggregator in communication with a plurality of components. The system can update, using the first application, the virtual register responsive to the aggregator being configured. The system can transmit, using a second application, a second serial communication to at least one of the plurality of components responsive to the virtual register having been updated.
In various examples, synthetic data generation, reinforcement learning, and supervised learning may each run at scale within corresponding clusters of compute nodes, the compute nodes in each cluster may use specialized hardware tailored to a corresponding task, and the tasks running the different clusters may collectively operate in a loop or circular workflow using the output of one task as the input to the next. For example, parallel synthetic data generation may continuously generate synthetic data, parallel supervised learning jobs may continuously apply the latest set of skills or experiences learned during reinforcement learning to generate an updated model using the latest synthetic data as training data, and parallel reinforcement learning jobs may continuously use the latest version of the model to learn new skills or experiences. The present techniques may be used to train and/or test humanoid robots, physical AI, or other AI systems and applications.
Supervised learning and synthetic data generation may be implemented at scale using a hub-and-spoke model of data centers or clusters in which all spoke data centers connect and share data via high bandwidth connections to a networked file system in the hub data center. For example, multiple instances of a (e.g., robotics) foundation model may be trained in one or more training clusters (e.g., in a hub data center) using synthetic video generated by multiple instances of a simulation environment (e.g., NVIDIA Isaac Sim) in parallel in one or more simulation clusters (e.g., in one or more spoke data centers). Parallel synthetic data generation may continuously generate synthetic data, and parallel supervised learning jobs may continuously generate an updated model using the latest synthetic data as training data. The present techniques may be used to train and/or test humanoid robots, physical AI, or other AI systems and applications.
Reinforcement and supervised learning may be implemented at scale using a hub-and-spoke model of data centers or clusters in which all spoke data centers connect and share data via high bandwidth connections to a networked file system in the hub data center. For example, instances of a (e.g., robotics foundation) model may be trained in training cluster(s) (e.g., in a hub data center) using skills or experiences learned using reinforcement learning running in instances of a simulation environment in simulation cluster(s) (e.g., in spoke data center(s)). Parallel supervised learning jobs may continuously apply the latest set of skills or experiences learned during reinforcement learning to generate an updated model, and parallel reinforcement learning jobs may continuously use the latest version of the model to learn new skills or experiences. The present techniques may be used to train and/or test humanoid robots, physical AI, or other AI systems and applications.
In various examples, geometry model-based three-dimensional reconstruction for interior space monitoring systems and applications are provided. An interior geometry model may be used to reconstruct a 3D geometry of an interior by selecting distinct regions of interest corresponding to interior surface regions. A point cloud representation of one or more selected regions may be generated using the vehicle interior geometry and a set of commonly oriented 3D bounding shapes (e.g., bounding boxes generated with respect to a shared coordinate system) may be generated to bound the selected interior surfaces. Bounded point cloud samples may be projected onto a surface using a surface fitting algorithm to estimate a best non-circumscribed polygon region model, which provides a geometric framework for reconstructing a 3D geometry that may be used to obtain direct measurements of 3D relationships between the selected regions of interest and OMS sensors.
In various examples, data processing using inter-chip communication for computing systems and applications is described herein. Systems and methods described herein may use interfaces that transmit data between chips to perform one or more processing tasks, such as image stitching, image cropping, format conversion, and/or any other processing task. For instance, image data obtained using image or camera sensors may be stored in source buffers of a first chip. The image data may then be associated with descriptors used to transmit the image data from the source buffers to a destination buffer of a second chip. For instance, a descriptor may indicate at least an identifier of a source buffer, an address within the source buffer, an address within the destination buffer, and a length of data being transmitted. As described herein, in some examples, transmitting the image data using the descriptors may cause the processing task(s) to be performed.
Designing binding proteins is a central challenge in computational biology. Modern artificial intelligence (AI)-based approaches fall into two classes: (1) generative methods which treat binder design as conditional generation, training on binder-target complex structures and producing new candidates for unseen targets; and (2) hallucination methods which use the confidence and alignment scores of structure predictors to assess interfaces and optimize binder sequences via gradient feedback. However, these current approaches do not unify the generative and hallucination methods. The present disclosure provides synthetic protein binder generation conditioned on a given target protein via a framework that unifies a flow-based base generative model with flexible inference-time optimization utilizing the generative prior, thereby combining the strengths of both approaches.
In various examples, updates to a dynamic seam placement and/or fitted 3D bowl may be at least partially concealed using spatial masking. A future time in which a predicted change in dynamic seam placement and/or fitted 3D bowl exceeds some threshold may be determined, and a predicted dynamic seam movement and/or fitted 3D bowl update may be spatially masked by triggering a viewport switch to coincide with (a) the predicted dynamic seam placement and/or fitted 3D bowl update and/or (b) a relaxation or disabling of temporal filtering. Additionally or alternatively to predicting that a future change will exceed a threshold, the determination of the change may occur based on a change between a current and previous frame. In some embodiments that employ viewport switching to spatially mask visualization updates, the switch may be to one of a plurality of candidate viewports for an applicable scene maintained in a scene catalog.
Video-based multimodal large language models that include an image encoder configured to transform frames from a video into visual tokens, a multimodal large language model configured to receive the visual tokens, and a temporal module interposed between the image encoder and the multimodal large language model.
In various examples, lane biasing for navigating around objects in autonomous systems and applications is described. Systems and methods are disclosed that generate lane (or other demarcated regions of an environment) geometries in environments—such as environments without clear lane or boundary demarcations—by using locations and poses of static objects and/or outputs of a drivable free-space analysis. The systems and methods then use the lane geometries, the locations of the static objects, and current paths (e.g., centerlines of the current paths) along the lanes to determine new paths for navigating around the static objects. For instance, the new paths may be determined by shifting the centerlines of the current paths in directions away from the static objects by some distance or safety margin. This way, the vehicles are able to navigate around the static objects.
Disclosed are systems and techniques for rendering digital avatar animations. The techniques include receiving a user input associated with an interaction of a user with a digital avatar, identifying, based at least on the user input, a state of the interaction, and obtaining, based at least on the state, one or more animations and one or more configuration settings corresponding to the one or more animations. The techniques further include generating a blended animation based at least on the one or more animations and the one or more configuration settings for the one or more animations, rendering a temporal sequence of frames based at least on the blended animation, and causing a presentation of the temporal sequence of frames.
Apparatuses, systems, and techniques to optimize processor performance. In at least one embodiment, a method increases a maximum operating voltage (Vmax) of one or more processors to be dynamically adjusted, based at least in part, on one or more indications of processor usage.
An integrated circuit includes an analog-to-digital converter (ADC) to determine a present power supply value of a power supply voltage for a thermal sensor. A memory stores calibration data associating thermal sensor outputs with calibrated temperature values for each of a plurality of power supply values. Control logic reads the present power supply value from the ADC and compensates a thermal sensor output based on the present power supply value and the calibration data to determine a calibrated temperature value.
Embodiments relate to hazard detection in autonomous and semi-autonomous systems and applications. A transformer may use sampled image and LiDAR features to extract and decode a representation of whether there is a hazard at the 3D location corresponding to each initial transformer query, the shape of the hazard, and/or its class. These detections may be provided to one or more control components of an autonomous vehicle, which may use the detections to navigate, plan, or otherwise perform one or more operations (e.g., obstacle avoidance, lane keeping, lane changing, merging, splitting, etc.). Some embodiments employ an automated approach to derive ground truth data from sensor data collected by data collection vehicle(s), such as data representing detected static scene points, navigable space boundaries, or detected hazard objects. Accordingly, hazards such as road debris and other obstacles may be detected and ground truth data may be generated for a variety of sensing tasks.
G01S 17/931 - Lidar systems, specially adapted for specific applications for anti-collision purposes of land vehicles
G01S 7/4865 - Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
G01S 17/86 - Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
G01S 17/894 - 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
G06V 10/26 - Segmentation of patterns in the image fieldCutting or merging of image elements to establish the pattern region, e.g. clustering-based techniquesDetection of occlusion
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestriansRecognition of traffic objects, e.g. traffic signs, traffic lights or roads
74.
LOW POWER PROXIMITY-BASED PRESENCE DETECTION USING OPTICAL FLOW
In various examples, low power proximity based threat detection using optical flow for vehicle systems and applications are provided. Some embodiments may use a tiered framework that uses sensor fusion techniques to detect and track the movement of a threat candidate, and perform a threat classification and/or intent prediction as the threat candidate approaches approach. Relative depth indications from optical flow, computed using data from image sensors, can be used to initially segment and track a moving object over a sequence of image frames. Additional sensors and processing may be brought online when a moving object becomes close enough to be considered a higher risk threat candidate. A threat response system may generate a risk score based on a predicted intent of a threat candidate, and when the risk score exceeds a certain threshold, then the threat response system may respond accordingly based on the threat classification and/or risk score.
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestriansRecognition of traffic objects, e.g. traffic signs, traffic lights or roads
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
G06V 20/59 - Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
G08B 13/196 - Actuation by interference with heat, light, or radiation of shorter wavelengthActuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
75.
System and method for runtime optimization of neural networks
Systems and methods of optimizing runtime of a neural network (NN) by at least one processor may include: receiving a space of untrained NN architectures, capable of performing a predefined NN function on a given computing device, after being trained; receiving a reference accuracy metric value; evaluating a latency value for each NN architecture of the architecture space; dividing the architecture space to a plurality of groups based on said evaluated latency; performing a search among the plurality of groups, to determine a group G* that corresponds to a minimal evaluated latency, and yet comprises at least one NN architecture that maintains an accuracy metric value that is at least equal to the reference accuracy metric value; and training at least one NN architecture of the determined group G* to perform the NN function.
Apparatuses, systems, and techniques to perform an application programming interface (API) to indicate one or more graph node functions of one or more graph nodes to be added to a software graph based, at least in part, on a dependency type indicated by the API. In at least one embodiment, one or more graph nodes are added to a software graph based on a node type and a dependency type.
In various examples, natural speech audio may be generated from transcript text and a representation of context such as an audio sample or text description of a target speaker. For example, a language model implemented using multiple non-autoregressive transformer encoders may be used to generate separate embeddings of transcript text and context, each of which may be applied to separate decoder layers to constrain cross-attention over the transcript text to be monotonic and leave cross-attention over the context unconstrained. In some embodiments, the language model may support multiple context encoders corresponding to different context modalities (e.g., an audio clip of a reference speaker, a textual description of a reference speaker, an audio clip of a conversation history), such that any supported context modality may be applied during deployment, and the appropriate context encoder may be activated and used to synthetize and output a corresponding speech waveform.
Apparatuses, systems, methods, and techniques to perform special functions. In at least one embodiment, values input into a special function unit (SFU) have a first data format (e.g., a fixed point data format) that is different from a second data format (e.g., a fixed point data format) of values output by the SFU. In at least one embodiment, a plurality of special function units (SFUs) are arranged in parallel, and used to perform special functions based at least in part on portions of an input vector. In at least one embodiment, quantizer(s) convert values in the input vector to a fixed point data format. In at least one embodiment, portions of an output vector are obtained based at least part on output of the SFUs.
Apparatuses, systems, methods, and techniques to perform graphics processing unit (GPU) initiated networking (GIN). In at least one embodiment, generation of a work queue element (WQE) is offloaded from the GPU to separate hardware (e.g., an accelerator or processor in a network interface controller). In at least one embodiment, the GPU generates a GPU-friendly descriptor (GFD), instead of a WQE, that contains information to create a WQE and transfers the GFD to the separate hardware, which performs the creation of the WQE, processes a completion queue entry (CQE), and provides a notification to the GPU that processing the WQE has been completed.
Technologies for providing precise peak-to-peak amplitude detection for harmonic-rich circuits are described. A peak-to-peak amplitude detector circuit can include a peak-to-peak rectifier and a linear combiner. The peak-to-peak rectifier can rectify a positive peak and a negative peak of an incoming analog signal to obtain a positive and a negative rectified signal. The linear combiner can combine and compare the positive and negative rectified signals and positive and negative target signals to obtain an output signal indicative of a peak-to-peak amplitude of the incoming analog signal.
H03K 5/24 - Circuits having more than one input and one output for comparing pulses or pulse trains with each other according to input signal characteristics, e.g. slope, integral the characteristic being amplitude
81.
NEURAL DENOISING OF PATH TRACED IMAGES WITH SCREEN SPACE EFFECTS
In various examples, one or more processors can include one or more circuits to obtain an image generated by executing path tracing operations. The one or more circuits can apply an effect to the image in two-dimensional screen space and can update the image to represent a difference between the original image and the image, with the effect applied to improve downstream denoising of the image. The updated image can then be provided to a denoiser that implements one or more neural networks that are trained/updated to output a final image that is denoised and displayed using a display device.
In various examples, a technique for performing spatial mapping and planning with large language models using knowledge graphs may include querying a temporal knowledge graph to predict a next step that a robotic system is to traverse in an environment, wherein the temporal knowledge graph comprises a graph representation of entities and relationships in the environment, wherein the entities and relationships in the graph representation are produced based on sensor data captured from the environment. The technique also may include receiving additional sensor data captured from the environment. The technique further may include generating, via a machine learning model, spatial and temporal data associated with the additional sensor data. The technique still further may include updating, via the machine learning model, the temporal knowledge graph to include representations of the spatial and temporal data, based on a similarity between the spatial and temporal data and the graph representation.
Systems and methods are disclosed that process a pool of incoming requests using a mixture of expert large language model (MOE-LLM). For instance, the pool of incoming requests may be sorted into a plurality of groups of requests based on using a scheduler that selects one or more subsets of a plurality of feedforward layers (FFNs) from the MOE-LLM to service the plurality of groups of requests. The first subset of the plurality of FFNs are loaded into memory of the GPU based on a first group of requests. The first group of requests are serviced using the first subset of the plurality of FFNs to generate responses. While servicing the first group of requests, a second subset of the plurality of FFNs are loaded into the memory of the GPU based on a second group of requests that are to be serviced after the first group of requests.
Retrieving data blocks from a memory cache on a processor unit to be used by more than one compute thread can be improved to increase data reuse and reduce power consumption. Conventionally, data blocks are retrieved using a standard data retrieval model regardless of how the data is reused by the multiple compute threads accessing that data. By dynamically using a combination of dataflow retrieval models, a more optimized process can be implemented increasing data reuse and lowering power consumption of the processor unit. The dataflow retrieval models can be adaptive swizzling, continuous rasterization, alternating k-order, periodic compute thread array synchronization, or explicit tile eviction. As the size and number of memory caches increase on a processing unit, as well as the number of logic units and streaming multiprocessors, these optimizations become more valuable to overall efficiency.
Systems and methods are provided for automatically modifying nodes in a computing cluster while maintaining workload continuity in the computing cluster. At least one embodiment relates to an operator that coordinates with a cluster manager to monitor nodes of the computing cluster, apply modifications to the nodes of the computing cluster, and schedule workloads for the nodes of the computing cluster based on a custom resource.
A method, computer readable medium, and system are disclosed for action video generation. The method includes the steps of generating, by a recurrent neural network, a sequence of motion vectors from a first set of random variables and receiving, by a generator neural network, the sequence of motion vectors and a content vector sample. The sequence of motion vectors and the content vector sample are sampled by the generator neural network to produce a video clip.
G06N 3/044 - Recurrent networks, e.g. Hopfield networks
G06N 3/0442 - Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
In various examples, systems and methods are disclosed relating to generating an output 3D latent representation by encoding, using a text encoder, a text prompt and encoding, using a 2D-3D encoder, a 2D image of an object or a 3D representation of the object. A 3D output is generated by applying the output 3D latent representation to a decoder. A reconstruction loss and a SDS loss are determined for the 3D output. At least one of the text encoder, the 2D-3D encoder, and the decoder is updated using the reconstruction loss and the SDS loss.
Apparatuses, systems, and methods to use one or more neural networks to generate information about one or more images based, at least in part, on one or more confidence scores associated with the information. In at least one embodiment, a neural network downscales an image and performs an image processing task on said downscaled image according to a query input by a user.
G06T 3/4046 - Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
G06V 10/26 - Segmentation of patterns in the image fieldCutting or merging of image elements to establish the pattern region, e.g. clustering-based techniquesDetection of occlusion
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
89.
APPLICATION PROGRAMMING INTERFACE TO MODIFY TENSOR DIMENSIONS
Apparatuses, systems, and techniques to perform image processing. In at least one embodiment, an application programming interface is performed to cause dimensions of a tensor to be modified.
Apparatuses, systems, and techniques to automatically detect stale data in accelerator-enabled programs. In at least one embodiment, a runtime monitors a software program that has been segmented between accelerated and non-accelerated portions and, during debugging, identifies when various memory locations associated with the accelerated and non-accelerated code segments are read or written to, and if their values are out-of-sync.
Embodiments of the present disclosure relate to automated tuning of control parameters of a controller. In some implementations, one or more parameter sets with at least one changing variable and that determine how the controller responds to an environment may be obtained. In these and other implementations, at least one of the parameter sets may include a vector parameter that includes a vector of values. In some implementations, the vector of values for the vector parameter may be ordered and at least one operation of the controller may be simulated using the the ordered vector of values for the vector parameter.
G05B 13/04 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric involving the use of models or simulators
A system may identify lane lines (e.g., from sensor data) for autonomous vehicle control. The approach works by detecting individual lane line segments and identifying center lines that run along the length of each segment. By connecting these center lines, the system reconstructs complete lane lines. The identified lane lines then inform control operations for the machine. The sensor data may come from cameras, LIDAR, or RADAR. The system can be implemented in various forms, including systems-on-chip, autonomous vehicle control systems, or cloud-based platforms. The machine performing these operations may be autonomous or semi-autonomous, to perceive lane markings and navigate accordingly.
G06V 10/34 - Smoothing or thinning of the patternMorphological operationsSkeletonisation
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
G06V 20/56 - Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
G06V 20/58 - Recognition of moving objects or obstacles, e.g. vehicles or pedestriansRecognition of traffic objects, e.g. traffic signs, traffic lights or roads
93.
DENOISING PATH-TRACED IMAGES USING TRANSFORMER NETWORKS
In various examples, one or more processors can include one or more circuits to obtain image data for a current image being generated, in accordance with one or more ray tracing operations. The image data can correspond to a current image prior to or during reconstruction of that image. The image can be grainy, include artifacts, etc. To reduce these affects, the one or more circuits can execute one or more attention operations to determine an update to the current image based at least on a plurality of patches of the current image and windowed attention regions corresponding to each patch of the plurality of patches. In some examples, the windowed attention regions can extend at least in part beyond boundaries of each patch, allowing for information from neighboring patches to be used to update the image when reconstructing the image.
Systems and methods disclosed herein can be used to provide power for a non-powered rack chassis. An enclosure shaped to fit within the non-powered rack chassis may be associated with or may include a power shelf. The power shelf may have at least one busbar to distribute the power from the power shelf to one or more sleds of the non-powered rack chassis.
Technologies for correcting transmitter common mode (CM) voltage deviations in serial data are described. A transmitter device includes a clock source to supply a plurality of phase signals, and a resonant structure to receive the plurality of phase signals and generate a plurality of sinusoidal phase signals. The transmitter device also includes a first serializer to receive parallel data and convert the parallel data to serial data using the plurality of sinusoidal phase signals, and a transmitter coupled to an output of the first serializer. The transmitter device also includes a duty-cycle correction circuit coupled ton an output node of the transmitter. The duty-cycle correction circuit can detect a CM voltage deviation in the serial data, and correct the CM voltage deviation by applying a voltage offset to the plurality of sinusoidal phase signals.
Approaches presented herein include an audio generation system that incorporates a two-part generator having an encoder and decoder structure. An input mel-spectrogram is provided to the encoder to generate embeddings for the decoder to upsample and then produce one or more waveforms. During training, a discriminator may be used to evaluate the one or more waveforms to update weights of the encoder and/or the decoder. To conserve memory consumption, gradient checkpoints associated with the decoder may be deleted upon generation of the outputs and then, during backpropagation, gradients may be recomputed.
G10L 21/0332 - Details of processing therefor involving modification of waveforms
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
97.
MULTI-DECODER SPEECH RECOGNITION MODELS WITH MULTI-FORMAT OUTPUTS
Disclosed are apparatuses, systems, and techniques for generating multi-format transcriptions of speech. The techniques include processing, using an encoder, audio frames representative of a speech to generate embeddings encoding the speech and processing, using multiple decoders, the embeddings to generate multiple transcriptions of the speech. An individual transcription is generated by a respective decoder and conforms to a respective text format that differs from other text formats in capitalization, punctuation, use of non-alphabet characters, and/or identification of individual utterances of the speech.
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
Systems and methods disclosed relate to generating training data. In one embodiment, the disclosure relates to systems and methods for generating training data to train a neural network to detect and classify objects. A simulator obtains 3D models of objects, and simulates 3D environments comprising the objects using physics-based simulations. The simulations may include applying real-world physical conditions, such as gravity, friction, and the like on the objects. The system may generate images of the simulations, and use the images to train a neural network to detect and classify the objects from images.
In machine learning, attention mechanisms are used to determine the importance of each component in a sequence relative to the other components in that sequence. Attention mechanisms enable artificial intelligence models to focus on the most relevant parts of data to provide context-aware outputs, which improves tasks such as machine translation, text summarization, image captioning, speech recognition, etc. Traditional attention models compute attention for multi-dimensional data for every possible pair of data elements in the multi-dimensional data or sequentially element-by-element, both approaches of which are computationally expensive and time-consuming. The present disclosure provides an attention mechanism that employs multi-dimensional linear propagation when computing attention information, which can improve spatial coherence and efficiency by directly operating on spatially coherent image data and forming dense pairwise connections through the multi-dimensional linear approach.
G06V 10/764 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
In various examples, a VPU and associated components may be optimized to improve VPU performance and throughput. For example, the VPU may include a min/max collector, automatic store predication functionality, a SIMD data path organization that allows for inter-lane sharing, a transposed load/store with stride parameter functionality, a load with permute and zero insertion functionality, hardware, logic, and memory layout functionality to allow for two point and two by two point lookups, and per memory bank load caching capabilities. In addition, decoupled accelerators may be used to offload VPU processing tasks to increase throughput and performance, and a hardware sequencer may be included in a DMA system to reduce programming complexity of the VPU and the DMA system. The DMA and VPU may execute a VPU configuration mode that allows the VPU and DMA to operate without a processing controller for performing dynamic region based data movement operations.
