METHOD AND APPARATUS FOR SUPPORTING DISTRIBUTED GRAPHICS AND COMPUTE ENGINES AND SYNCHRONIZATION IN MULTI-DIELET PARALLEL PROCESSOR ARCHITECTURES -- MEMORY BARRIERS
This disclosure describes supporting distributed graphics and compute engines in a multi-dielet processor, such as, for example, a multi-dielet graphics processing unit (GPU), architectures and synchronization in such architectures. Each multi-dielet processor includes a hardware-implemented remapping capability and/or a hardware-implemented memory barrier capability.
One embodiment of a method for controlling a robot includes receiving sensor data associated with an environment that includes an object; applying a machine learning model to a portion of the sensor data associated with the object and one or more trajectories of motion of the robot to determine one or more path lengths of the one or more trajectories; generating a new trajectory of motion of the robot based on the one or more trajectories and the one or more path lengths; and causing the robot to perform one or more movements based on the new trajectory.
In various examples, providing virtual assistants for content streaming systems and applications is described herein. For instance, systems and methods are disclosed that use a virtual assistant associated with an application, such as a gaming application, to at least process queries received from a user in order to provide the user with information on how to perform various tasks associated with the application. In some examples, to determine the output information, data associated with the application is processed in order to determine state information describing a current state of the application. Additionally, the query, the state information, and/or additional information may be used to determine contextual information related to the query. One or more language models may then process the query and/or the information to determine the output information associated with the query. The output information may then be provided using various techniques, such as text, graphics, and/or audio.
In various examples, policy prediction-based motion planner systems and methods for autonomous and semi-autonomous systems and applications are provided. A scenario tree structure may be generated that represents potential behaviors of one or more peripheral agents based on perception data of a scene within which an ego vehicle operates. A joint MPC algorithm may optimize the motion of an ego vehicle within the context of the scenario tree structure to produce a policy tree structure. An MPC policy prediction model may be trained to predict the policy tree structures that a joint MPC algorithm would produce, given a set of environmental perception data. An ego vehicle may comprise a trained MPC policy prediction model that receives perception data, and based on that input predicts a policy tree structure that may be used to define a motion policy for navigating the ego vehicle through the scene.
In various examples, systems and techniques are provided that encapsulate indexing and query operations into an application programming interface (API) that automates and coordinates calls to various local and cloud-based services. When a user has a document(s) to add to a retrieval augmented generation (RAG) database, the API may offer to the user multiple document processing pipelines (DPPs) having pre-set indexing configurations. Similarly, when a user query is received, the API may generate calls to implement query processing that does not require the user to manually configure retrieval and processing of the embeddings. The API may further implement calls that locate a relevant embedding store and provide the stored embeddings, together with the query embeddings, to a search engine that identifies the most relevant matches. The API may then access the embedding-to-text indexing and identify relevant text segments and documents to a prompt generator.
In various examples, an environment visualization pipeline may determine whether to generate or otherwise enable a visualization using an environmental modeling pipeline that models an environment as a 3D bowl or using an environmental modeling pipeline that models the environment using some other 3D representation, such as a detected 3D surface topology. The determination may made based on various factors, such as ego-machine state, (e.g., one or more detected features indicative of a designated operational scenario, proximity to a detected object, speed of ego-machine, etc.), estimated image quality of a corresponding environment visualization, and/or other factors. Accordingly, an environment around an ego-machine, such as a vehicle, robot, and/or other type of object, may be visualized in systems such as parking visualization systems, Surround View Systems, and/or others.
B60W 50/14 - Moyens d'information du conducteur, pour l'avertir ou provoquer son intervention
G06T 7/50 - Récupération de la profondeur ou de la forme
G06T 17/20 - Description filaire, p. ex. polygonalisation ou tessellation
G06V 10/26 - Segmentation de formes dans le champ d’imageDécoupage ou fusion d’éléments d’image visant à établir la région de motif, p. ex. techniques de regroupementDétection d’occlusion
G06V 10/98 - Détection ou correction d’erreurs, p. ex. en effectuant une deuxième exploration du motif ou par intervention humaineÉvaluation de la qualité des motifs acquis
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
7.
FEATURE GENERATION OF DASHED LINE COMPONENTS FOR AUTONOMOUS SYSTEMS AND APPLICATIONS
In various examples, systems and methods described herein may determine individual components of a dashed line based at least on identifying relationships across different portions of the dashed line. For instance, input data representing a road surface may be analyzed and a representation associated with a dashed line may be determined. In some instances, the representation may be generated based at least on intensity values associated with points corresponding to the input data. Then, based at least on the representation, information associated with one or more components of the dashed line may be determined. For instance, the representation may be indicative of the relationships across the different portions of the dashed line, and these relationships may be used to determine the information associated with the one or more components of the dashed line.
G06V 20/56 - Contexte ou environnement de l’image à l’extérieur d’un véhicule à partir de capteurs embarqués
G01S 17/89 - Systèmes lidar, spécialement adaptés pour des applications spécifiques pour la cartographie ou l'imagerie
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
G06V 10/77 - Traitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source
G06V 10/88 - Reconnaissance d’images ou de vidéos utilisant des moyens optiques, p. ex. filtres de référence, masques holographiques, filtres de domaine de fréquence ou filtres de domaine spatial
8.
Universal Scale Metadata Layout for Matrix Multiply and Add (MMA)
This disclosure describes efficiently performing matrix multiply and add (MMA) operations using narrow operands. Narrow operand size (e.g., 8 bit/6 bit/4 bit operand) MMA operations utilize scale metadata in order to improve accuracy of the MMA operation. An efficient layout for scale metadata in narrow operand size MMA operations and its use are described. The proposed layout provides for efficient storing and efficient use of scale metadata.
a first set of threads having a same address corresponding to the shared memory is identified from a group of active threads associated with an instruction to update a shared memory. A first thread of the first set of threads is selected. The instruction is executed for the first thread using the same address to access the shared memory. Attempts to execute the instruction for remaining threads of the first set of threads are delayed until after the first thread is executed and until at least one of the remaining threads of the first set of threads is not guaranteed to fail execution of the instruction.
Some embodiments relate to environmental text perception using vision language models (VLMs). For example, an Advanced Driver Assistance System (ADAS) may identify candidate parking spaces, and a VLM may be used to evaluate parking signs and determine whether it is permissible and/or the cost to park in a candidate parking space. For example, frames from corresponding (e.g., front-facing, repeater, side pillar) camera(s) may be evaluated for corresponding parking signs (e.g., using a sign recognition DNN or a VLM). If a parking sign is detected, the image of the sign may be provided as input to a VLM with a textual prompt instructing the VLM to determine whether it is permissible to park at a corresponding location (and if so, the cost). The generated response may be provided to the ADAS to confirm or invalidate the candidate parking space, and a representation of the results may be provided to the driver.
B60W 50/14 - Moyens d'information du conducteur, pour l'avertir ou provoquer son intervention
G06Q 30/0283 - Estimation ou détermination de prix
G06V 20/58 - Reconnaissance d’objets en mouvement ou d’obstacles, p. ex. véhicules ou piétonsReconnaissance des objets de la circulation, p. ex. signalisation routière, feux de signalisation ou routes
11.
HARDWARE-BASED INTER-PROCESSING COMMUNICATION NETWORKS FOR MANAGED DEVICES
Disclosed are apparatuses, systems, and techniques that implement software-agnostic transport of messages to, from, and within managed devices. In one embodiment, a managed device has an intra-device network including a plurality of units, each unit associated with a unit controller. The managed device further includes a hub controller that receives data packet(s) jointly carrying a message from an external host. The controller identifies that the one or more first data packets are associated with a given unit and forwards the data packet(s) to the corresponding unit controller. The unit controller extracts the message from the data packet(s) and stores the message in a memory associated with the unit controller.
Some embodiments relate to driver or occupant monitoring using vision language models (VLMs). Any number of DNNs in a detection pipeline may be replaced with a VLM, and the VLM may be prompted to determine whether a corresponding feature is present in an image or sampled frames from a video. To facilitate using the VLM(s) to control one or more downstream actions, the VLM(s) may be prompted using structured inputs, and a designated output format for a corresponding structured output may be enforced in any suitable manner. As such, any number of VLMs may be used to perform any number of driver and/or occupant monitoring tasks (e.g., driver drowsiness detection, driver distraction detection, driver or occupant out-of-position detection, driver or occupant identification, seatbelt usage detection, occupant presence detection, occupant classification, child presence detection, gesture recognition, occlusion detection, and/or others).
G06V 20/59 - Contexte ou environnement de l’image à l’intérieur d’un véhicule, p. ex. concernant l’occupation des sièges, l’état du conducteur ou les conditions de l’éclairage intérieur
B60W 50/14 - Moyens d'information du conducteur, pour l'avertir ou provoquer son intervention
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
In photorealistic image synthesis by light transport simulation, the colors of each pixel are computed by evaluating an integral of a high-dimensional function. In practice, the pixel colors are estimated by using Monte Carlo and quasi-Monte Carlo methods to sample light transport paths that connect light sources and cameras and summing up the contributions to evaluate the integral. Because of the sampling, images appear noisy when the number of samples is insufficient. Due to the lack of information, denoising the shaded images introduces artifacts, for example, blurred the images. Denoising before material shading enables real-time light transport simulation, producing high visual quality even for low sampling rates (avoiding the blurred shading). The light transport integral operator is evaluated by a neural network, requiring data from only a single frame.
Systems, computer program products, and methods are described for a hybrid quantum-classical system for enhanced combinatorial optimization. An example system segments a received task into multiple sub-tasks. For each sub-task, the system accesses a database of pre-computed solutions through the classical computing unit to identify a suitable pre-computed solution. In scenarios where a pre-computed solution is not available for a sub-task, the classical computing unit transmits this sub-task to a quantum computing unit. The computing unit, utilizing a quantum optimization algorithm, computes a solution for the sub-task. This solution is then relayed back to the classical computing unit. The classical computing unit then implements each identified pre-computed and newly computed solution on the combinatorial optimization task.
Techniques for training a machine learning model to control a robot include performing, based on a first set of data, one or more training operations to generate a first trained machine learning model to control a robot and a trained evaluation model, and performing, based on a second set of data and first feedback generated by the trained evaluation model, one or more training operations to generate a second trained machine learning model to control the robot, where the second set of data is associated with a different set of sensor modalities than the first set of data.
This disclosure describes supporting distributed graphics and compute engines in a multi-dielet parallel processing system, such as, for example, a multi-dielet graphics processing unit (GPU), architectures and synchronizing memory management in such architectures. Respective dielets each has a memory management unit (MMU). The processing of at least one memory-related message type is serialized by a designated MMU for messages originated at any dielet, and the processing of at least some memory-related message types is performed locally on the originating dielets.
G06F 12/1027 - Traduction d'adresses utilisant des moyens de traduction d’adresse associatifs ou pseudo-associatifs, p. ex. un répertoire de pages actives [TLB]
G06F 12/1009 - Traduction d'adresses avec tables de pages, p. ex. structures de table de page
In various examples, image data may be received that represents an image. Corner detection may be used to identify pixels that may be candidate corner points. The image data may be converted from a higher dimensional color space to a converted image in a lower dimensional color space, and boundaries may be identified within the converted image. A set of the candidate corner points may be determined that are within a threshold distance to one of the boundaries, and the set of the candidate corner points may be analyzed to determine a subset of the candidate corner points representative of corners of polygons. Using the subset of the candidate corner points, one or more polygons may be identified, and a filter may be applied to the polygons to identify a polygon as corresponding to a fiducial marker boundary of a fiducial marker.
In various examples, three-dimensional multi-camera perception systems and applications is described herein. Systems and methods are disclosed herein that process image data generated using multiple cameras located throughout an environment in order to directly determine three-dimensional (3D) information associated with objects located within the environment. For instance, the image data may be processed using one or more feature extractors (e.g., one or more backbones) to determine multi-view image features associated with images represented by the image data. These multi-view image features, along with calibration data associated with the cameras, may then be processed using one or more spatio-temporal transformers (e.g., one or more spatial encoders, one or more temporal encoders, etc.) in order to determine 3D locations of objects within the environment.
In various examples, systems and methods are disclosed relating to geometry estimation and dynamic object rendering for vehicle environment visualization. In embodiments, the environment surrounding an ego-machine may be visualized by extracting one or more depth maps from image data, converting the depth map(s) into a 3D surface topology of the surrounding environment, and/or texturizing the detected 3D surface topology with image data. Dynamic objects such as moving vehicles or pedestrians may be detected and masked from a first pass of texturization. Rigid dynamic objects may be visualized by warping corresponding depth values using corresponding trajectories, inserting or fusing the resulting warped 3D representation of each such object into the (e.g., texturized) 3D surface topology, and texturizing the warped 3D representation of each object using corresponding image data. Non-rigid dynamic objects may be represented as flat 2D surfaces and texturized with corresponding image data.
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
20.
THREE-DIMENSIONAL (3D) HEAD POSE PREDICTION FOR AUTOMOTIVE SYSTEMS AND APPLICATIONS
In various examples, head pose prediction for automotive occupant sensing systems and applications is presented. The systems and methods described herein provide for a machine learning model trained using a dataset that comprises ground truth head pose data computed using a registered head model of a training subject. While operating a vehicle, one or more cameras and a depth sensor capture synchronized images of the training subject. To compute a ground truth 3D head pose, angular deviations between a 3D point cloud and the registered head model may be computed to obtain a 3D ground truth head pose measurement. Using an extrinsic calibration transform, the head pose measurement may be mapped into the sensor coordinate frame. Training samples may be produced for training the machine learning model that comprise an optical image frame and the head pose measurement transposed into the frame of reference for that optical image frame.
In some embodiments, the same vision language model (VLM) may be used to support different types of detection tasks (e.g., one foundational VLM supporting some or all detection tasks performed by an ego-machine, one VLM for interior sensing tasks and one for exterior sensing tasks, etc.), and an inference scheduler may be used to serve or handle inference requests for the VLM(s) to perform the different tasks. In some embodiments, the scheduler prioritizes inference requests based on safety (e.g., prioritizing inference requests to perform ADAS tasks such as pedestrian detection, bicycle detection, or trajectory planning over requests to perform driver or occupant monitoring tasks, prioritizing exterior sensing tasks over interior sensing tasks, etc.). As such, the scheduler may queue, manage, distribute inference requests from different detection applications to the VLM(s), and receive and return responses to corresponding detection task managers.
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/776 - ValidationÉvaluation des performances
22.
MACHINE LEARNING MODELS FOR RECONSTRUCTION AND SYNTHESIS OF DYNAMIC SCENES FROM VIDEO
In various examples, systems and methods are disclosed relating to reconstruction and synthesis of dynamic scenes from video, such as to generate a four-dimensional (4D) representation of one or more scenes based on one or more videos (e.g., two-dimensional (2D) videos) of the one or more scenes. A system may determine, using a neural network and based on a three-dimensional (3D) representation of one or more scenes, a 4D representation of the one or more scenes, the 3D representation generated by a featurizer using a plurality of first image frames from video data of the one or more scenes. The system may determine, from the 4D representation, a target image having a target pose and a target time.
G06T 17/00 - Modélisation tridimensionnelle [3D] pour infographie
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/44 - Extraction de caractéristiques locales par analyse des parties du motif, p. ex. par détection d’arêtes, de contours, de boucles, d’angles, de barres ou d’intersectionsAnalyse de connectivité, p. ex. de composantes connectées
23.
METHOD FOR FORWARD PROGRESS AND PROGRAMMABLE TIMEOUTS OF TREE TRAVERSAL MECHANISMS IN HARDWARE
In a ray tracer, to prevent any long-running query from hanging the graphics processing unit, a traversal coprocessor provides a preemption mechanism that will allow rays to stop processing or time out early. The example non-limiting implementations described herein provide such a preemption mechanism, including a forward progress guarantee, and additional programmable timeout options that can be time or cycle based. Those programmable options provide a means for quality of service timing guarantees for applications such as virtual reality (VR) that have strict timing requirements.
In various examples, various types of sensor data from multiple ego-machines are used to infer lanes and/or generate lane graphs for use in autonomous systems and applications. In some embodiments, one or more DNNs may be used to infer lane data indicating a representation of a lane shape using sensor data from various vehicles to represent a 3D environment. The inferred lane data may include cross-section indicators that indicate cross-sections of a lane and/or connection indicators that indicate a lane channel connecting two locations (e.g., two lane portions). The inferred lane data may be used to generate a lane graph that represents lanes on a road and, in some cases, lane dividers (e.g., polyline represented as a solid line, a dashed line, a double line, etc.). A lane graph may be used, for example, to model the environment around a vehicle, facilitate localization, provide guidance for autonomous driving, etc.
In various examples, various types of sensor data from multiple ego-machines are used to infer lanes and/or generate lane graphs for use in autonomous systems and applications. In some embodiments, one or more DNNs may be used to infer lane data indicating a representation of a lane shape using sensor data from various vehicles to represent a 3D environment. The inferred lane data may include cross-section indicators that indicate cross-sections of a lane and/or connection indicators that indicate a lane channel connecting two locations (e.g., two lane portions). The inferred lane data may be used to generate a lane graph that represents lanes on a road and, in some cases, lane dividers (e.g., polyline represented as a solid line, a dashed line, a double line, etc.). A lane graph may be used, for example, to model the environment around a vehicle, facilitate localization, provide guidance for autonomous driving, etc.
Computing system performance monitors provide on-chip control, selection, collection, coalescing and communication of behavior and other processing-indicating data of high performance single- and multi-die computing and processing systems, such as for use in multi-chip-module and/or multi-instanced graphics processing units (GPUs) and/or systems-on-chips (SOCs). Commands and data records can be forwarded between modules to abstract the processing system from profilers and other data report consumers. Quality of Service and security isolation for different command and data report streams is maintained.
Disclosed are devices, systems, and techniques for evaluation of machine learning models, pipelines of machine learning models, retrieval-augmented generation (RAG) systems, and/or other artificial intelligence systems. Example techniques include receiving, from a client device, an evaluation task to evaluate a language model (LM) using a plurality of evaluation benchmarks (EBs) associated with respective EB dataset and configuring, using an evaluation API, respective sets of evaluation jobs to implement the evaluation task. An individual set of evaluation jobs is configured to evaluate, using the corresponding EB dataset, performance of the LM to obtain a set of evaluation metrics. The techniques further include executing the sets of evaluation jobs to obtain respective sets of evaluation metrics and causing, using the evaluation API, a representation of the sets of evaluation metrics to be provided to the client device.
Disclosed are devices, systems, and techniques for evaluation of machine learning models, pipelines of machine learning models, retrieval-augmented generation (RAG) systems, and/or other artificial intelligence systems. Example techniques include receiving, from a client device, an evaluation task to evaluate a language model (LM) using a plurality of evaluation benchmarks (EBs) associated with respective EB dataset and configuring, using an evaluation API, respective sets of evaluation jobs to implement the evaluation task. An individual set of evaluation jobs is configured to evaluate, using the corresponding EB dataset, performance of the LM to obtain a set of evaluation metrics. The techniques further include executing the sets of evaluation jobs to obtain respective sets of evaluation metrics and causing, using the evaluation API, a representation of the sets of evaluation metrics to be provided to the client device.
H04L 41/22 - Dispositions pour la maintenance, l’administration ou la gestion des réseaux de commutation de données, p. ex. des réseaux de commutation de paquets comprenant des interfaces utilisateur graphiques spécialement adaptées [GUI]
A first request to allocate one or more memory blocks of a first plurality of memory blocks associated with a first memory is received by a processing device. A consecutive set of a first portion of bits of a first register with a first logical state is identified. The first logical state indicates that corresponding memory blocks of the one or more memory blocks are free. A first operation to adjust the consecutive set of the first portion of bits of the first register to a second logical state is performed. An allocation address comprising an index of the consecutive set of the first portion of bits of the first register is sent to the first request. The allocation address is useable to access the corresponding memory blocks.
Disclosed are devices, systems, and techniques for training, deployment, inference, benchmarking, and evaluation of machine learning models. Example techniques include receiving a selection of a first task for a large language model (LLM) and instantiating an execution container including one or more compute backends. The example techniques further include receiving, via an evaluation API, task data into the execution container, the task data having one or more LLM prompts. The example techniques further include executing, using the compute backend(s), the first task in the execution container to generate a task output that includes one or more LLM responses to the LLM prompt(s) or a modification of parameters of the LLM based at least on the LLM prompt(s). The example techniques further include evaluating, using evaluation benchmarks accessed by the evaluation API, the task output to obtain metrics characterizing performance of the LLM and executing a second task using the LLM.
In various examples, a language model may be trained and used as part of an interface for a simulation system. For instance, user inputs may be applied to the language model and the language model may be trained to generate code, make API calls, or perform any other operations to interact with and/or control various aspects of the simulation. In some examples, the language model may generate code for, among other things, creating and/or customizing a virtual environment associated with the simulation. For instance, the generated code may include, but is not limited to, code for rendering the virtual environment, code for rendering and simulating behaviors of virtual agents (e.g., pedestrians, vehicles, animals, etc.) and/or any other objects (e.g., road signs, buildings, trees, etc.) within the virtual environment, code for recreating and simulating real-world events from recorded sensor data, etc.
This disclosure describes supporting distributed graphics and compute engines in a multi-dielet processor, such as, for example, a multi-dielet graphics processing unit (GPU), architectures and synchronization in such architectures. Each multi-dielet processor includes a hardware-implemented remapping capability and/or a hardware-implemented memory barrier capability.
G06F 15/80 - Architectures de calculateurs universels à programmes enregistrés comprenant un ensemble d'unités de traitement à commande commune, p. ex. plusieurs processeurs de données à instruction unique
33.
ADDRESS TRANSLATION SERVICES TO ENABLE MEMORY COHERENCE
A first virtual address is translated into a first physical address using a first translation agent associated with a first I/O device of a system. The first physical address is associated with an address space of the first I/O device. A first address translation request is sent to a second translation agent associated with a CPU of the system. The first address translation request includes the first physical address. A first address translation response is received from the second translation agent. The second address translation response includes a second physical address. the second physical address is associated with an address space of the system.
G06F 12/0815 - Protocoles de cohérence de mémoire cache
G06F 12/1027 - Traduction d'adresses utilisant des moyens de traduction d’adresse associatifs ou pseudo-associatifs, p. ex. un répertoire de pages actives [TLB]
G06T 1/20 - Architectures de processeursConfiguration de processeurs p. ex. configuration en pipeline
Techniques for improving the accuracy of delay line calibration schemes. For example, an amount of offset may be determined between one or more first portions of a first clock signal and one or more second portions of a second clock signal that is delayed relative to the first clock signal. The first portion(s) may correspond to the second portion(s) based at least on the second clock signal being delayed relative to the first clock signal. In some examples, a value may be determined based at least on the amount of offset. The value may correspond to an amount to adjust the first clock signal to reduce the amount of offset. In some examples, a delay line may then be calibrated, based at least on the second value, to adjust the first clock signal.
H03L 7/081 - Détails de la boucle verrouillée en phase avec un déphaseur commandé additionnel
G06F 1/08 - Générateurs d'horloge ayant une fréquence de base modifiable ou programmable
G06F 1/12 - Synchronisation des différents signaux d'horloge
H03K 5/14 - Dispositions ayant une sortie unique et transformant les signaux d'entrée en impulsions délivrées à des intervalles de temps désirés par l'utilisation de lignes à retard
35.
MULTIMODAL LARGE LANGUAGE MODEL AGENT FOR AUTONOMOUS DRIVING
Multimodal large language models (MLLMs) have excellent reasoning capabilities and are used for autonomous driving applications. For real-world applications, understanding and navigating in three-dimensional (3D) space is necessary, particularly for autonomous vehicles (AVs) to make informed decisions, anticipate future states, and interact safely with the environment. An MLLM agent system includes a 3D projector model and an adapted LLM that extends understanding and reasoning capability from 2D to 3D. Another component of the MLLM agent system is development of a benchmark visual question-answering (VQA) training dataset for training the MLLM agent. The VQA tasks include scene description, traffic regulation, 3D grounding, counterfactual reasoning, decision making, and planning.
A hardware mechanism at each dielet of a multi-dielet processing system is aware of engine page-table binds at all the dielets, thereby providing accurate traffic notifications to software (e.g., a unified virtual memory driver) for on-demand page-migration between system memory and GPU memory. The mechanism broadcasts binding information to access counters on each dielet so the access counters are able to correlate engines requesting memory access with bound virtual memory pages and generate corresponding informative notifications. A flexible multi-dielet counter clear capability enables software to clear access counters.
In various examples, multi-sensor subject tracking for monitored environments for real-time and near-real-time systems and applications are provided. A location system performs multi-subject tracking using streaming data from multiple sensors. Subject tracking may be based on individual anchors and behavior states that are initialized for individual subjects using representations (e.g., behavior embeddings) derived from the streaming data. Clustering may be used to generate behavior clusters that individually represents a trackable subject. Behavior states for live anchors may identified based on continuity of trajectory and tracked by iteratively propagating their behavior states forward over time. Clusters lacking continuity of trajectory may be used to initialize new anchors, or matched to dormant anchors that may be reclassified as live anchors and propagated. Propagated behavior states may be updated using behavior data represented by the behavior embeddings.
G06V 10/762 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant le regroupement, p. ex. de visages similaires sur les réseaux sociaux
G06V 40/20 - Mouvements ou comportement, p. ex. reconnaissance des gestes
38.
DIRECT CONNECT BETWEEN NETWORK INTERFACE AND GRAPHICS PROCESSING UNIT IN SELF-HOSTED MODE IN A MULTIPROCESSOR SYSTEM
Various embodiments include techniques for performing data transfer operations via a direct interconnect between a network interface and a graphics processor in a multiprocessor system that also includes a central processing unit (CPU). The CPU communicates with the graphics processor via a dedicated high-bandwidth interconnect to the memory in the graphics processor and a second interconnect to the graphics processor for various utility functions. The network interface communicates with the graphics processor via an interconnect to the memory in the graphics processor. The interconnect between the network interface and the graphics processor does not impact the throughput of the high-bandwidth interconnect from the CPU to the graphics processor, thereby improving CPU to graphics processor performance. Further, the interconnect between the CPU to the graphics processor does not impact the throughput of the interconnect from the network interface to the graphics processor, thereby improving network interface to graphics processor performance.
Approaches presented herein provide for the testing of imaging algorithms and systems. In at least one embodiment, a stereoscopic test pattern can be obtained that includes a number of features that vary in width and separation, such as may comprise a set of radial elements that converge toward a center point. A stereoscopic image of an instance of the pattern can be analyzed, such as at a set of radial positions, to make various measurements, including a limit on the ability to distinguish between different features. A pair of synthetic images of the pattern can be generated in order to test aspects of a stereoscopic algorithm used to generate stereoscopic images, with such testing being separate from the physical system, and a physical object can be generated that includes a representation of the pattern in order to be able to test the physical stereoscopic imaging system.
H04N 13/239 - Générateurs de signaux d’images utilisant des caméras à images stéréoscopiques utilisant deux capteurs d’images 2D dont la position relative est égale ou en correspondance à l’intervalle oculaire
Apparatuses, systems, and techniques are to transfer data based, at least in part, on a computational graph. In at least one embodiment, a processor causes a compiler to generate instructions to prefetch one or more data values from dynamic random access memory (DRAM) into an in-processor cache based, at least in part, on a computational graph.
G06F 12/0862 - Adressage d’un niveau de mémoire dans lequel l’accès aux données ou aux blocs de données désirés nécessite des moyens d’adressage associatif, p. ex. mémoires cache avec pré-lecture
G06F 12/1072 - Traduction d’adresse décentralisée, p. ex. dans des systèmes de mémoire partagée distribuée
41.
APPLICATION PROGRAMMING INTERFACE TO INDICATE DEVICE ATTRIBUTE
Apparatuses, systems, and techniques to perform an application programming interface (API) to indicate whether one or more processors are able to be controlled by two or more drivers concurrently. An API is performed that will indicate whether a compute driver and a graphics driver can concurrently control a processor.
In various examples, one or more Machine Learning Models (MLMs) are used to identify content items in a video stream and present information associated with the content items to viewers of the video stream. Video streamed to a user(s) may be applied to an MLM(s) trained to detect an object(s) therein. The MLM may directly detect particular content items or detect object types, where a detection may be narrowed to a particular content item using a twin neural network, and/or an algorithm. Metadata of an identified content item may be used to display a graphical element selectable to acquire the content item in the game or otherwise. In some examples, object detection coordinates from an object detector used to identify the content item may be used to determine properties of an interactive element overlaid on the video and presented on or in association with a frame of the video.
G06V 20/40 - ScènesÉléments spécifiques à la scène dans le contenu vidéo
A63F 13/537 - Commande des signaux de sortie en fonction de la progression du jeu incluant des informations visuelles supplémentaires fournies à la scène de jeu, p. ex. en surimpression pour simuler un affichage tête haute [HUD] ou pour afficher une visée laser dans un jeu de tir utilisant des indicateurs, p. ex. en montrant l’état physique d’un personnage de jeu sur l’écran
H04L 65/611 - Diffusion en flux de paquets multimédias pour la prise en charge des services de diffusion par flux unidirectionnel, p. ex. radio sur Internet pour la multidiffusion ou la diffusion
43.
ENVIRONMENTAL TEXT PERCEPTION AND TOLL EVALUATION USING VISION LANGUAGE MODELS
A vision language model (VLM) may be used to evaluate signs that designate restricted or toll lanes, determine whether it is permissible (and/or the cost) to merge into a restricted or toll lane, and/or determine when to merge out of a restricted or toll lane based on the cost. Frames from one or more (e.g., front-facing) camera(s) may be evaluated for applicable signs (e.g., using a sign recognition DNN or a VLM). If detected, the (e.g., cropped) image of the sign may be provided as input to a VLM with a textual prompt instructing the VLM to determine whether to drive in the restricted or toll lane (e.g., whether it can be taken within budget) and/or what the cost would be. The generated response may be provided to an ADAS to trigger an initiation of a merge left or right or a determination to stay in the current lane.
G01C 21/34 - Recherche d'itinéraireGuidage en matière d'itinéraire
G06V 20/58 - Reconnaissance d’objets en mouvement ou d’obstacles, p. ex. véhicules ou piétonsReconnaissance des objets de la circulation, p. ex. signalisation routière, feux de signalisation ou routes
Apparatuses, systems, and techniques are to transfer data based, at least in part, on a computational graph. In at least one embodiment, a processor causes a compiler to generate instructions to prefetch one or more data values from dynamic random access memory (DRAM) into an in-processor cache based, at least in part, on a computational graph.
A processing device including a first cache is coupled to a system memory and a parallel processing unit (PPU) including a second cache. An operation to modify cache lines of the second cache associated with a first aperture of the system memory is received. A first subset of cache lines of the second cache is identified. The first subset of cache lines is associated with the first aperture of the system memory and is different from a second subset of cache lines of a second aperture of the system memory. The first subset of cache lines is modified as specified by the cache operation.
A first instruction to load a first data into a first register file associated with a first subpartition unit is received. A second instruction to load the first data into a second register file associated with a second subpartition unit is received. The first instruction and the second instruction are coalesced into a first entry of a request coalescer based on instruction identifiers. The first entry is associated with the first data. Responsive to a determination that the first data is available in the cache, the first data is multicast from the cache to the first register file and the second register file.
G06F 12/0804 - Adressage d’un niveau de mémoire dans lequel l’accès aux données ou aux blocs de données désirés nécessite des moyens d’adressage associatif, p. ex. mémoires cache avec mise à jour de la mémoire principale
G06F 12/0868 - Transfert de données entre une mémoire cache et d'autres sous-systèmes, p. ex. des dispositifs de stockage ou des systèmes hôtes
G06F 12/0875 - Adressage d’un niveau de mémoire dans lequel l’accès aux données ou aux blocs de données désirés nécessite des moyens d’adressage associatif, p. ex. mémoires cache avec mémoire cache dédiée, p. ex. instruction ou pile
Apparatuses, systems, and techniques to perform signal processing operations in a fifth generation (5G) new radio (NR) signal. In at least one embodiment, one or more processors process a 5G NR signals according to one or more graph nodes.
H04W 72/0446 - Ressources du domaine temporel, p. ex. créneaux ou trames
G06F 16/901 - IndexationStructures de données à cet effetStructures de stockage
H04B 7/26 - Systèmes de transmission radio, c.-à-d. utilisant un champ de rayonnement pour communication entre plusieurs postes dont au moins un est mobile
H04L 5/00 - Dispositions destinées à permettre l'usage multiple de la voie de transmission
H04W 72/0453 - Ressources du domaine fréquentiel, p. ex. porteuses dans des AMDF [FDMA]
In various examples, methods and systems are provided for estimating depth values for images (e.g., from a monocular sequence). Disclosed approaches may define a search space of potential pixel matches between two images using one or more depth hypothesis planes based at least on a camera pose associated with one or more cameras used to generate the images. A machine learning model(s) may use this search space to predict likelihoods of correspondence between one or more pixels in the images. The predicted likelihoods may be used to compute depth values for one or more of the images. The predicted depth values may be transmitted and used by a machine to perform one or more operations.
G06T 7/55 - Récupération de la profondeur ou de la forme à partir de plusieurs images
G06T 7/70 - Détermination de la position ou de l'orientation des objets ou des caméras
G06V 10/46 - Descripteurs pour la forme, descripteurs liés au contour ou aux points, p. ex. transformation de caractéristiques visuelles invariante à l’échelle [SIFT] ou sacs de mots [BoW]Caractéristiques régionales saillantes
49.
MOTION VECTOR OPTIMIZATION FOR MULTIPLE REFRACTIVE AND REFLECTIVE INTERFACES
Systems and methods relate to the determination of accurate motion vectors, for rendering situations such as a noisy Monte Carlo integration where image object surfaces are at least partially translucent. To optimize the search for “real world” positions, this invention defines the background as first path vertices visible through multiple layers of refractive interfaces. To find matching world positions, the background is treated as a single layer morphing in a chaotic way, permitting the optimized algorithm to be executed only once. Further improving performance over the prior linear gradient descent, the present techniques can apply a cross function and numerical optimization, such as Newton's quadratic target or other convergence function, to locate pixels via a vector angle minimization. Determined motion vectors can then serve as input for services including image denoising.
Systems and methods for determining the gaze direction of a subject and projecting this gaze direction onto specific regions of an arbitrary three-dimensional geometry. In an exemplary embodiment, gaze direction may be determined by a regression-based machine learning model. The determined gaze direction is then projected onto a three-dimensional map or set of surfaces that may represent any desired object or system. Maps may represent any three-dimensional layout or geometry, whether actual or virtual. Gaze vectors can thus be used to determine the object of gaze within any environment. Systems can also readily and efficiently adapt for use in different environments by retrieving a different set of surfaces or regions for each environment.
G06F 18/21 - Conception ou mise en place de systèmes ou de techniquesExtraction de caractéristiques dans l'espace des caractéristiquesSéparation aveugle de sources
G06F 18/214 - Génération de motifs d'entraînementProcédés de Bootstrapping, p. ex. ”bagging” ou ”boosting”
G06V 10/764 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant la classification, p. ex. des objets vidéo
G06V 10/774 - Génération d'ensembles de motifs de formationTraitement des caractéristiques d’images ou de vidéos dans les espaces de caractéristiquesDispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant l’intégration et la réduction de données, p. ex. analyse en composantes principales [PCA] ou analyse en composantes indépendantes [ ICA] ou cartes auto-organisatrices [SOM]Séparation aveugle de source méthodes de Bootstrap, p. ex. "bagging” ou “boosting”
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/94 - Architectures logicielles ou matérielles spécialement adaptées à la compréhension d’images ou de vidéos
G06V 20/59 - Contexte ou environnement de l’image à l’intérieur d’un véhicule, p. ex. concernant l’occupation des sièges, l’état du conducteur ou les conditions de l’éclairage intérieur
Denoising images rendered using Monte Carlo sampled ray tracing is an important technique for improving the image quality when low sample counts are used. Ray traced scenes that include volumes in addition to surface geometry are more complex, and noisy when low sample counts are used to render in real-time. Joint neural denoising of surfaces and volumes enables combined volume and surface denoising in real time from low sample count renderings. At least one rendered image is decomposed into volume and surface layers, leveraging spatio-temporal neural denoisers for both the surface and volume components. The individual denoised surface and volume components are composited using learned weights and denoised transmittance. A surface and volume denoiser architecture outperforms current denoisers in scenes containing both surfaces and volumes, and produces temporally stable results at interactive rates.
Enhanced techniques applicable to a ray tracing hardware accelerator for traversing a hierarchical acceleration structure and its underlying primitives are disclosed. For example, traversal speed is improved by grouping processing of primitives sharing at least one figure (e.g., a vertex or an edge) during ray-primitive intersection testing. Grouping the primitives for ray intersection testing can reduce processing (e.g., projections and transformations of primitive vertices and/or determining edge function values) because at least a portion of the processing results related to the shared feature in one primitive can be used to determine whether the ray intersects another primitive(s). Processing triangles sharing an edge can double the culling rate of the triangles in the ray/triangle intersection test without replicating the hardware.
Apparatuses, systems, and techniques to perform a neural network to perform an API to cause storage to be reserved. In at least one embodiment, for example, an API causes storage to be reserved based, at least in part, on a flag indicating a memory pool to be allocated to a memory of a processor, such as a GPU or CPU. In at least one embodiment, as another example, a processor comprising one or more circuits performs an application programming interface (API) to indicate whether one or more graphics processing units (GPU) are to access GPU storage or host storage.
Approaches presented herein provide systems and methods for dynamic allocation of processing units to increase computational density. Idle times between sequential processing tasks may be computed and, if the idle time exceeds a threshold capacity, additional sequential processing tasks may be allocated to a common processing unit. As a request, a first portion of a first sequential processing task may be executed, then a second portion of a second sequential processing task may be executed prior to executing a subsequent portion of the second sequential processing task. By using the idle time between portions of sequential processing tasks, output perform may be maintained while using additional processing capabilities that would otherwise remain idle.
Disclosed are apparatuses, systems, and techniques for implementing bidirectional tracking in computer vision applications. In one embodiment, the techniques include obtaining digital representations of an object depicted in video frames and for each of a forward direction (FD) of tracking and a reverse direction (RD) of tracking, obtaining, using (i) a current state of the object associated with an upstream video frame and (ii) the digital representation of the object for a downstream video frame, an updated state of the object associated with the downstream video frame. The techniques further include obtaining, using the updated state of the object for the FD and/or the updated state of the object for the RD, a bidirectional state of the object, and determining, using the bidirectional state of the object, a trajectory of the object across the video frames.
G06T 7/73 - Détermination de la position ou de l'orientation des objets ou des caméras utilisant des procédés basés sur les caractéristiques
G06V 10/74 - Appariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
56.
SPEED DETERMINATION IN ROBOTICS SYSTEMS AND APPLICATIONS
In various examples, a technique for generating speed change decisions for a mobile robot includes identifying, using one or more maps of a physical environment, one or more obstacles associated with one or more portions of a path of the mobile robot in the physical environment. The technique also includes generating, based at least on the one or more obstacles, one or more speed constraints, each speed constraint specifying a speed limit for a respective portion of the path. The technique further includes generating one or more speed change decisions specifying actions to be performed by the mobile robot to cause a speed profile of the mobile robot to satisfy the one or more speed constraints.
Approaches presented herein provide systems and methods to selectively apply packet loss mitigation methods to one or more regions of a frame that have a sufficient importance value. The importance value may be determined by a saliency map generated for the frame that determine the most important content elements or regions of the frame. An importance value may be computed based on the saliency map and then, for areas of sufficient importance, selective mitigation methods may be used to reduce bandwidth, conserve compute resources, and provide error correction or duplication for important regions of the frame.
H04N 21/238 - Interfaçage de la voie descendante du réseau de transmission, p. ex. adaptation du débit de transmission d'un flux vidéo à la bande passante du réseauTraitement de flux multiplexés
H04N 21/234 - Traitement de flux vidéo élémentaires, p. ex. raccordement de flux vidéo ou transformation de graphes de scènes du flux vidéo codé
H04N 21/24 - Surveillance de procédés ou de ressources, p. ex. surveillance de la charge du serveur, de la bande passante disponible ou des requêtes effectuées sur la voie montante
A capacitive monitoring structure includes a ring oscillator and dynamically configurable capacitive load circuits coupled between stages of the ring oscillator. The oscillation frequency of the ring oscillator changes in response to settings applied to the capacitive load circuits to change a capacitance applied to the ring oscillator by the capacitive load circuits, where the capacitance may be one of a transistor drain capacitance, a transistor gate capacitance, and a Miller capacitance.
G01R 27/26 - Mesure de l'inductance ou de la capacitanceMesure du facteur de qualité, p. ex. en utilisant la méthode par résonanceMesure de facteur de pertesMesure des constantes diélectriques
H01L 27/092 - Transistors à effet de champ métal-isolant-semi-conducteur complémentaires
59.
LOCAL TRANSFORM PROPAGATION IN ENVIRONMENT RECONSTRUCTION SYSTEMS AND APPLICATIONS
Approaches presented herein provide for the matching and alignment of features in different instances of sensor data corresponding to an environment. At least one embodiment provides for accurate identification of matching lane dividers between two or more tracks obtained from sensor-equipped vehicles or machines. An initial transform can be determined using a seed area for tracks of data, where the seed area can be determined using landmarks, lane boundaries, or other such objects identified from the sensor data. The initial transform can be used to determine lane divider matches in the track data. If successfully evaluated, these lane divider matches from the seed areas can be propagated out in one or more tracking directions along a roadway to determine lane divider matches along entire stretches of roadway, including roads that pass through intersections or other relatively complex regions.
In various examples, each hosted application may be modeled with a corresponding application-specific resource consumption model that predicts a measure of that application's anticipated resource utilization at some future time based on an input representation of one or more features of the current state of an instance of the hosted application. For cloud gaming, those features may include the current level being played, current obstacles, user results playing the level or obstacles, metadata quantifying one or more aspects of the level or obstacles, game progress, etc. As such, application-specific models may be used to predict resource demands at a future time and schedule resource allocations accordingly. The present techniques may be used to manage and reallocate resources for applications such as game streaming applications, remote desktop applications, simulation applications (e.g., an autonomous or semi-autonomous vehicle simulation), virtual reality (VR) and/or augmented reality (AR) streaming applications, and/or other application types.
Approaches presented herein can provide for the performance of specific types of tasks using a large model, without a need to retrain the model. Custom endpoints can be trained for specific types of tasks, as may be indicated by the specification of one or more guidance mechanisms. A guidance mechanism can be added to or used along with a request to guide the model in performing a type of task with respect to a string of text. An endpoint receiving such a request can perform any marshalling needed to get the request in a format required by the model, and can add the guidance mechanisms to the request by, for example, prepending one or more text strings (or text prefixes) to a text-formatted request. A model receiving this string can process the text according to the guidance mechanisms. Such an approach can allow for a variety of tasks to be performed by a single model.
Approaches presented herein can provide for the performance of specific types of tasks using a large model, without a need to retrain the model. Custom endpoints can be trained for specific types of tasks, as may be indicated by the specification of one or more guidance mechanisms. A guidance mechanism can be added to or used along with a request to guide the model in performing a type of task with respect to a string of text. An endpoint receiving such a request can perform any marshalling needed to get the request in a format required by the model, and can add the guidance mechanisms to the request by, for example, prepending one or more text strings (or text prefixes) to a text-formatted request. A model receiving this string can process the text according to the guidance mechanisms. Such an approach can allow for a variety of tasks to be performed by a single model.
Apparatuses, systems, and techniques to scale processor clocks. In at least one embodiment, one or more circuits are to scale one or more clocks of one or more cores based, at least in part, on a proximity of the one or more cores to each other.
Apparatuses, systems, and techniques to cause one or more neural networks to summarize a text. In at least one embodiment, a processor is to cause one or more neural networks to generate one or more summaries of a first portion of a text based, at least in part, on one or more second portions of said text.
In various examples, a technique for generating a path between a current location and a target waypoint is disclosed that includes receiving a route plan that is associated with a plurality of waypoints representing locations in a physical environment. The technique also includes identifying a search space that includes the route plan, and identifying a target waypoint of the plurality of waypoints—the target waypoint being in a portion of the search space between a current location of a mobile robot and an end waypoint of the route plan. A path between the current location of the mobile robot and the target waypoint may then be generated.
In various examples, systems and methods are disclosed relating to health and error monitoring of sensor fusion systems. Systems and methods are disclosed that aggregate results of monitoring and error checking in a sensor fusion system in a single checkpoint. A processor may include one or more circuits. The one or more circuits may receive perception data from one or more first sensors of a machine. The one or more circuits may receive position data from one or more second sensors of the machine. The one or more circuits may generate output data by performing fusion of at least the perception data and the position data. The one or more circuits may evaluate a plurality of criteria according to at least a subset of the perception data, the position data, and the output data. The one or more circuits may output an error signal according to the evaluation.
G06F 11/36 - Prévention d'erreurs par analyse, par débogage ou par test de logiciel
G06F 18/21 - Conception ou mise en place de systèmes ou de techniquesExtraction de caractéristiques dans l'espace des caractéristiquesSéparation aveugle de sources
Apparatuses, systems, and techniques of autonomously adjust priority of information to be transmitted by a UE device. In at least one embodiment, a UE device autonomously adjusts priority of information to be transmitted by adjusting prioritization parameters. In at least one embodiment, prioritization parameters are adjusted autonomously by a UE device that monitors packet statistics.
Various embodiments include techniques for processing memory operations in a computing system. The computing system includes a central processing unit (CPU) and an auxiliary processor, such as a parallel processing unit (PPU). The PPU can be divided into multiple partitions. Although the partitions are included in a single PPU, the CPU can track the partitions as if the partitions are independent devices rather than different portions of a single device. When two different partitions generate memory operations that access the same memory address in CPU memory address space, the two partitions employ two different data paths. The CPU can use path information for the two different paths to identify which partition generated each memory operation. As a result, the CPU can maintain data consistency and memory coherency in a system where a PPU is divided into multiple partitions.
G06F 12/0831 - Protocoles de cohérence de mémoire cache à l’aide d’un schéma de bus, p. ex. avec moyen de contrôle ou de surveillance
G06F 12/0895 - Mémoires cache caractérisées par leur organisation ou leur structure de parties de mémoires cache, p. ex. répertoire ou matrice d’étiquettes
69.
Audio noise removal using one or more neural networks
Apparatuses, systems, and techniques are presented to reduce noise in audio. In at least one embodiment, a sequence of neural networks is used to remove foreground and background noise from audio including a primary audio signal.
G10L 21/00 - Techniques de traitement du signal de parole ou de voix pour produire un autre signal audible ou non audible, p. ex. visuel ou tactile, afin de modifier sa qualité ou son intelligibilité
G06N 3/044 - Réseaux récurrents, p. ex. réseaux de Hopfield
G10L 15/16 - Classement ou recherche de la parole utilisant des réseaux neuronaux artificiels
G10L 15/22 - Procédures utilisées pendant le processus de reconnaissance de la parole, p. ex. dialogue homme-machine
G10L 21/0232 - Traitement dans le domaine fréquentiel
G10L 25/18 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes caractérisées par le type de paramètres extraits les paramètres extraits étant l’information spectrale de chaque sous-bande
G10L 25/84 - Détection de la présence ou de l’absence de signaux de voix pour différencier la parole du bruit
70.
GRID-BASED LIGHT SAMPLING FOR RAY TRACING APPLICATIONS
Devices, systems, and techniques to incorporate lighting effects into computer-generated graphics. In at least one embodiment, a virtual scene comprising a plurality of lights is rendered by subdividing the virtual area and stored, in a record corresponding to a subdivision of the virtual area, information indicative of one or more lights in the virtual area selected based on a stochastic model. Pixels near a subdivision are rendered based on the light information stored in the subdivision.
Systems and methods for cooling a datacenter are disclosed. In at least one embodiment, a liquid-to-air heat exchanger is associated with a fan wall and a refrigerant-based cooling system to provide air cooling and refrigerant-based cooling to cool secondary coolant or fluid received from at least one cold plate.
A ray (e.g., a traced path of light, etc.) is generated from an originating pixel within a scene being rendered. Additionally, one or more shadow map lookups are performed for the originating pixel to estimate an intersection of the ray with alpha-tested geometry within the scene. A shadow map stores the distance of geometry as seen from the point of view of the light, and alpha-tested geometry includes objects within the scene being rendered that have a determined texture and opacity. Further, the one or more shadow map lookups are performed to determine a visibility value for the pixel (e.g., that identifies whether the originating pixel is in a shadow) and a distance value for the pixel (e.g., that identifies how far the pixel is from the light). Further still, the visibility value and the distance value for the pixel are passed to a denoiser.
Approaches presented herein provide systems and methods for reading a portion of data from an unprocessed memory segment to a registry and changing a status identifier for the unprocessed memory segment indicative of the data being read to the registry. The data may then be written to a destination memory segment from the registry. If a corresponding status identifier for the destination memory segments meets a value indicative that the destination memory segment has not yet been read into the registry, it may be read into the registry before overwriting it, and recursively written into its own destination memory segment.
Various embodiments include techniques for performing memory synchronization operations between processors in a multiprocessor computing system. A first processor transfers data by issuing memory operations to store the data to a shared memory. The first processor issues an asynchronous release operation to a load store unit. In response, the load store unit issues a memory synchronization operation to ensure that the data associated with the memory operations is visible in the shared memory. While the asynchronous release operation is pending, the first processor is able to issue further instructions and perform other operations. When the data associated with the memory operations is visible in the shared memory, the memory synchronization operation completes and the load store unit writes a flag to a separate memory location. Upon detecting that the flag has been written, a second thread, and/or other threads, can reliably read the data stored in the shared memory.
Approaches presented herein provide for the acceleration of a human review process, such as the review of annotations generated by a human labeler. Annotations (at least partially) generated by a human reviewer can be provided as input to a machine learning model trained to infer a probability of the annotations including at least one error. Annotations with a low probability of including an error can be approved automatically, while annotations with a high probability (e.g., above a threshold) of including an error can be directed for human review. In order to keep the human reviewer engaged, artificial errors may be introduced at various times based on various engagement criteria.
Disclosed are apparatuses, systems, and techniques that use label-looping processing for efficient automatic speech recognition (ASR). The techniques include performing a plurality of iterations of an outer processing loop to identify content units (CUs) of a media item having multiple frames. An individual iteration of the outer processing loop includes updating, using a first neural network (NN) and identified non-blank CU, a state of the media item and performing one or more iterations of an inner processing loop. An individual iteration of the inner processing loop includes processing, using a second NN, the state of the media item and an individual frame to predict a CU associated with the individual frame. The iterations of the inner processing loop are performed until the predicted CU corresponds to a non-blank CU. The identified plurality of CUs is used to generate a representation of the media item.
In various embodiments, an inductor package comprises a first inductor that is arranged in a first orientation and produces a first magnetic flux in a first direction; and a second inductor that is arranged in a second orientation and produces a second magnetic flux in a second direction that at least partially cancels the first magnetic flux, where the first direction is opposite the second direction. In some embodiments, a printed circuit board assembly comprises a printed circuit board (PCB) layer, a first inductor that is arranged on the PCB layer at a first orientation and produces a first magnetic flux in a first direction, and a second inductor that is arranged on the PCB layer at a second orientation and produces a second magnetic flux in a second direction that at least partially cancels the first magnetic flux.
In various examples, one or more output channels of a deep neural network (DNN) may be used to determine assignments of obstacles to paths. To increase the accuracy of the DNN, the input to the DNN may include an input image, one or more representations of path locations, and/or one or more representations of obstacle locations. The system may thus repurpose previously computed information—e.g., obstacle locations, path locations, etc.—from other operations of the system, and use them to generate more detailed inputs for the DNN to increase accuracy of the obstacle to path assignments. Once the output channels are computed using the DNN, computed bounding shapes for the objects may be compared to the outputs to determine the path assignments for each object.
G05D 1/00 - Commande de la position, du cap, de l'altitude ou de l'attitude des véhicules terrestres, aquatiques, aériens ou spatiaux, p. ex. utilisant des pilotes automatiques
Circuits that include one or more transmission lines to propagate a signal through a serially-arranged plurality of repeaters, and one or more control circuits to propagate control pulses to the repeaters, wherein a timing and duration of the control pulses is configured to operate the repeaters in current-mode signaling (CMS) mode during a state transition of the signal at the repeaters and to operate the repeaters in voltage-mode signaling (VMS) mode otherwise.
Apparatuses, systems, and techniques are presented to train and utilize one or more neural networks. A denoising diffusion generative adversarial network (denoising diffusion GAN) reduces a number of denoising steps during a reverse process. The denoising diffusion GAN does not assume a Gaussian distribution for large steps of the denoising process and applies a multi-model model to permit denoising with fewer steps. Systems and methods further minimize a divergence between a diffused real data distribution and a diffused generator distribution over several timesteps. Accordingly, various embodiments may enable faster sample generation, in which the samples are generated from noise using the denoising diffusion GAN.
The disclosure provides a voltage detecting circuit that detects voltage increases and voltage decreases using a diode drop and voltage thresholds. The voltage detecting circuit, referred to as a voltage variation detector, uses the diode to maintain a differential between the voltage being monitored and a voltage threshold. When the diode is reversed bias, the voltage variation detector generates a detecting signal indicating the monitored voltage crossed the voltage threshold. In one example, the method includes: (1) detecting at least one transition of a voltage across a voltage threshold, wherein the detecting is based on a transistor diode being reversed biased, (2) generating a detection signal when the voltage crosses the voltage threshold, and (3) performing one or more actions in response to the detection signal.
G01R 19/165 - Indication de ce qu'un courant ou une tension est, soit supérieur ou inférieur à une valeur prédéterminée, soit à l'intérieur ou à l'extérieur d'une plage de valeurs prédéterminée
G01R 19/17 - Indication de ce qu'un courant ou une tension est, soit supérieur ou inférieur à une valeur prédéterminée, soit à l'intérieur ou à l'extérieur d'une plage de valeurs prédéterminée indiquant le nombre de fois que le phénomène se produit
H02H 3/20 - Circuits de protection de sécurité pour déconnexion automatique due directement à un changement indésirable des conditions électriques normales de travail avec ou sans reconnexion sensibles à un excès de tension
In various embodiments, an inductor package comprises a first portion of an inductor coil that runs substantially along a first axis and carries an electrical current along the first axis in a first direction to produce a first magnetic flux, and a second portion of the inductor coil that runs substantially along the first axis and carries the electrical current along the first axis in a second direction to produce a second magnetic flux that at least partially cancels the first magnetic flux, where the first direction is opposite the second direction.
Apparatuses, systems, and techniques to generate images of objects. In at least one embodiment, one or more neural networks are trained to identify one or more objects within one or more images, and the one or more neural networks are used to generate an image of one or more objects.
A system and method for generating a digital avatar from a two-dimensional input image in accordance with a machine learning models is provided. The machine learning models are generative adversarial networks trained to process a latent code into three-dimensional data and color data. A generative adversarial network (GAN) inversion optimization algorithm is run on the first machine learning model to map the input image to a latent code for the first machine learning model. The latent code is used to generate unstructured 3D data and color information. A GAN inversion optimization algorithm is then run on the second machine learning model to determine a latent code for the second machine learning model, based at least on the output of the first machine learning model. The latent code for the second machine learning model is then used to generate the data for the digital avatar.
In various examples, model-based processing to reduce reaction times for content streaming systems and applications is described herein. Systems and methods are disclosed that use one or more machine learning models to process image data representative of frames of an application, such as a gaming application, in order to generate updated imaged data representative of one or more updated frames that help reduce reaction times for users. For instance, the machine learning model(s) may update one or more visual characteristics associated with the frames, such as a contrast, a brightness, and/or a saturation associated with the frames. As described herein, the machine learning model(s) may be trained to update the frames in order to reduce the reaction times of users, such as by using one or more loss functions that measure loss in predicted reactions times and/or loss associated with visual characteristics of frames.
G06T 19/20 - Édition d'images tridimensionnelles [3D], p. ex. modification de formes ou de couleurs, alignement d'objets ou positionnements de parties
A63F 13/213 - Dispositions d'entrée pour les dispositifs de jeu vidéo caractérisées par leurs capteurs, leurs finalités ou leurs types comprenant des moyens de photo-détection, p. ex. des caméras, des photodiodes ou des cellules infrarouges
88.
TECHNIQUES FOR INTERPRETABLE CLASSIFICATION VIA MULTI-LEVEL CONCEPT PROTOTYPES
One embodiment of a method for classifying data includes processing the data via a trained machine learning model that includes a plurality of layers, where each layer generates one or more corresponding features, generating a first distribution of features based on the one or more corresponding features generated by each layer included in the plurality of layers, and determining a first class for the data based on a comparison of the first distribution of features with one or more predefined distributions of features that are associated with one or more classes.
G06V 10/82 - Dispositions pour la reconnaissance ou la compréhension d’images ou de vidéos utilisant la reconnaissance de formes ou l’apprentissage automatique utilisant les réseaux neuronaux
G06V 10/75 - Organisation de procédés de l’appariement, p. ex. comparaisons simultanées ou séquentielles des caractéristiques d’images ou de vidéosApproches-approximative-fine, p. ex. approches multi-échellesAppariement de motifs d’image ou de vidéoMesures de proximité dans les espaces de caractéristiques utilisant l’analyse de contexteSélection des dictionnaires
89.
MACHINE-LEARNING-BASED ARCHITECTURE SEARCH METHOD FOR A NEURAL NETWORK
In at least one embodiment, differentiable neural architecture search and reinforcement learning are combined under one framework to discover network architectures with desired properties such as high accuracy, low latency, or both. In at least one embodiment, an objective function for search based on generalization error prevents the selection of architectures prone to overfitting.
G06N 3/04 - Architecture, p. ex. topologie d'interconnexion
G05B 13/02 - Systèmes de commande adaptatifs, c.-à-d. systèmes se réglant eux-mêmes automatiquement pour obtenir un rendement optimal suivant un critère prédéterminé électriques
G06F 7/57 - Unités arithmétiques et logiques [UAL], c.-à-d. dispositions ou dispositifs pour accomplir plusieurs des opérations couvertes par les groupes ou pour accomplir des opérations logiques
G06N 3/063 - Réalisation physique, c.-à-d. mise en œuvre matérielle de réseaux neuronaux, de neurones ou de parties de neurone utilisant des moyens électroniques
Systems, devices, and methods for disaggregating networking components are provided. An example networking chassis includes a first disaggregated server device supported by the networking chassis that includes a first central processing unit (CPU) and a first graphics processing unit (GPU) coupled with the first CPU. The networking chassis further includes a first insertable switch module communicably coupled with the first disaggregated server device that includes first switching chipsets and a first fabric management controller coupled with the first switching chipsets. The first insertable switch module at least partially controls data transmission associated with the first disaggregated server device. The first GPU of the first disaggregated server device is isolated on the first disaggregated server device, supported on the first disaggregated server device in the absence of other GPUs, or is otherwise the only GPU on the first disaggregated server device so as to provide modularity in networking applications.
Visual language processors that include an image encoder configured to convert an image into a low-resolution feature map, a feature refinement network configured to upsample the low-resolution feature map into a high-resolution feature map, and a visual-language connector configured to map an image-level feature map and a region-level feature map both derived from the high-resolution feature map into an embedding space of a language encoder.
G06V 10/771 - Sélection de caractéristiques, p. ex. sélection des caractéristiques représentatives à partir d’un espace multidimensionnel de caractéristiques
G06V 20/70 - Étiquetage du contenu de scène, p. ex. en tirant des représentations syntaxiques ou sémantiques
92.
MULTICAST AND REFLECTIVE MEMORY BEHAVIOR FOR MEMORY MODEL CONSISTENCY
In various examples, a memory model may support multicasting where a single request for a memory access operation may be propagated to multiple physical addresses associated with multiple processing elements (e.g., corresponding to respective local memory). Thus, the request may cause data to be read from and/or written to memory for each of the processing elements. In some examples, a memory model exposes multicasting to processes. This may include providing for separate multicast and unicast instructions or shared instructions with one or more parameters (e.g., indicating a virtual address) being used to indicate multicasting or unicasting. Additionally or alternatively, whether a request(s) is processed using multicasting or unicasting may be opaque to a process and/or application or may otherwise be determined by the system. One or more constraints may be imposed on processing requests using multicasting to maintain a coherent memory interface.
G06F 12/1045 - Traduction d'adresses utilisant des moyens de traduction d’adresse associatifs ou pseudo-associatifs, p. ex. un répertoire de pages actives [TLB] associée à une mémoire cache de données
G06F 12/109 - Traduction d'adresses pour espaces adresse virtuels multiples, p. ex. segmentation
G06F 13/16 - Gestion de demandes d'interconnexion ou de transfert pour l'accès au bus de mémoire
A new level(s) of hierarchy—Cooperate Group Arrays (CGAs)—and an associated new hardware-based work distribution/execution model is described. A CGA is a grid of thread blocks (also referred to as cooperative thread arrays (CTAs)). CGAs provide co-scheduling, e.g., control over where CTAs are placed/executed in a processor (such as a GPU), relative to the memory required by an application and relative to each other. Hardware support for such CGAs guarantees concurrency and enables applications to see more data locality, reduced latency, and better synchronization between all the threads in tightly cooperating collections of CTAs programmably distributed across different (e.g., hierarchical) hardware domains or partitions.
The disclosure provides a cooling solution that evaluates the thermal environment of a computer component based on transient thermal responses of the computer component. The transient thermal responses are generated by measuring the temperature rise of the computer component over a designated amount of time for multiple “good” assemblies and multiple “bad” assemblies to determine a duration and allowable temperature rise needed to set a pass/fail criteria for different failure modes of cooling devices. A cooling device may not be operating as designed due to damage, needed maintenance, missing thermal interface material (TIM), improper installation, etc. From the transient thermal responses, a thermal problem, such as a malfunctioning fan, can be determined and a corrective action can be performed.
G01K 7/42 - Circuits pour la compensation de l’inertie thermiqueCircuits pour prévoir la valeur stationnaire de la température
G01M 99/00 - Matière non prévue dans les autres groupes de la présente sous-classe
G05B 19/404 - Commande numérique [CN], c.-à-d. machines fonctionnant automatiquement, en particulier machines-outils, p. ex. dans un milieu de fabrication industriel, afin d'effectuer un positionnement, un mouvement ou des actions coordonnées au moyen de données d'un programme sous forme numérique caractérisée par des dispositions de commande pour la compensation, p. ex. pour le jeu, le dépassement, le décalage d'outil, l'usure d'outil, la température, les erreurs de construction de la machine, la charge, l'inertie
H05K 7/20 - Modifications en vue de faciliter la réfrigération, l'aération ou le chauffage
95.
IMAGE STITCHING WITH COLOR HARMONIZATION FOR SURROUND VIEW SYSTEMS AND APPLICATIONS
In various examples, color statistic(s) from ground projections are used to harmonize color between reference and target frames representing an environment. The reference and target frames may be projected onto a representation of the ground (e.g., a ground plane) of the environment, an overlapping region between the projections may be identified, and the portion of each projection that lands in the overlapping region may be taken as a corresponding ground projection. Color statistics (e.g., mean, variance, standard deviation, kurtosis, skew, correlation(s) between color channels) may be computed from the ground projections (or a portion thereof, such as a majority cluster) and used to modify the colors of the target frame to have updated color statistics that match those from the ground projection of the reference frame, thereby harmonizing color across the reference and target frames.
A circuit includes a phase selector to generate an injection clock signal having an injection phase based on a phase of a digitally controlled oscillator clock signal generated within a phase-locking feedback loop. An injection-locked oscillator (ILO), coupled to an output of the phase selector, generates an ILO clock signal that is convertible to provide a feedback clock signal of the circuit. Logic, coupled between an output of the ILO and the phase selector, to, at each predetermined number of cycles of the DCO clock signal, cause the phase selector to output a phase shift in the injection clock signal that causes the ILO clock signal to comprise a rotated phase, relative to the injection phase, and that prevents a glitch in the injection clock signal.
H03K 5/00 - Transformation d'impulsions non couvertes par l'un des autres groupes principaux de la présente sous-classe
H03K 19/20 - Circuits logiques, c.-à-d. ayant au moins deux entrées agissant sur une sortieCircuits d'inversion caractérisés par la fonction logique, p. ex. circuits ET, OU, NI, NON
97.
PARTITION-AWARE BROADCAST OPERATION FILTERING IN A MULTIPROCESSOR SYSTEM
Various embodiments include techniques for processing broadcast operations in a computing system. Typically, broadcast operations are transmitted to all portions of a particular subsystem, such as all cache slices in a cache memory. As a result, a process that issues a broadcast operation can interfere with one or more other processes that access portions of the subsystem not assigned to the process. To prevent such interference, logic in the computing system filters broadcast operations so as to transmit the broadcast operation to only the relevant portions assigned to the process that issued the broadcast operation. The logic tracks acknowledgments from the relevant portions and, when all pending acknowledgments have been received, the logic transmits a single acknowledgement to the process that issued the broadcast operation. The logic is dynamically configurable such that the logic can change the portions of the subsystem assigned to each process as needed.
In various examples, determining emotional states for speech in conversational artificial intelligence (AI) and/or digital avatar systems and applications is descried herein. Systems and methods are disclosed that use one or more machine learning models to determine one or more emotional states associated with speech, where the machine learning model(s) may be trained using various processes. For instance, in some examples, the machine learning model(s) may be trained during a first training process to determine probabilities for distributions of values, where the distributions model different emotional states. For example, a distribution may include a first value for angry, a second value for happy, a third value for sad, and/or so forth. Additionally, or alternatively, in some examples, the machine learning model(s) may be trained during a second training process to more precisely determine the actual emotional states (and/or the probabilities) based on training data representing human feedback.
G06N 7/01 - Modèles graphiques probabilistes, p. ex. réseaux probabilistes
G06T 13/40 - Animation tridimensionnelle [3D] de personnages, p. ex. d’êtres humains, d’animaux ou d’êtres virtuels
G10L 15/06 - Création de gabarits de référenceEntraînement des systèmes de reconnaissance de la parole, p. ex. adaptation aux caractéristiques de la voix du locuteur
G10L 25/57 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour le traitement des signaux vidéo
G10L 25/63 - Techniques d'analyse de la parole ou de la voix qui ne se limitent pas à un seul des groupes spécialement adaptées pour un usage particulier pour comparaison ou différentiation pour estimer un état émotionnel
99.
SUPPLEMENTING SENSOR DATA FOR PROCESSING USING AI SYSTEMS AND APPLICATIONS
In various examples, processing sensor data using rankings for AI systems and applications is described herein. Systems and methods are disclosed that determine rankings for sensor data, where the rankings may then be used to process the sensor data. For instance, a device that generates sensor data using a sensor may determine rankings for various portions of the sensor data, such as by analyzing the sensor data and/or related sensor data to detect configured events. For example, if the sensor data includes image data, then the device may determine a respective ranking for different groups of frames that are associated with different events. A system(s) that then use the rankings when processing the sensor data using one or more processing tasks. For example, the system(s) may determine which processing tasks to use for processing different portions of the sensor data based at least on the rankings.
Apparatuses, systems, and techniques are to represent polygon data as pixels as part of a rasterization process. In at least one embodiment, a processor causes identification of pixels within a polygon based, at least in part, on one or more prefix sums of amounts of edges of pixels. covered by that polygon.
