Technology is described for referencing anatomical structure using an AR headset. The method can include registering a marker located on a distal anatomical structure. The marker may have a pose that includes a position and orientation in a 3D coordinate system of the AR headset. Another operation may be identifying a proximal anatomical structure to which the distal anatomical structure is connected. The marker may be registered at a second pose having a second position and second orientation. A joint pivot axis and angle may be determined with respect to the proximal anatomical structure by comparing the first pose and the second pose. The joint pivot axis may then be displayed, using the AR headset. The technology may also use a pointer device to automatically find an anatomic path and anatomic length.
G06T 19/00 - Manipulating 3D models or images for computer graphics
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
G06V 10/24 - Aligning, centring, orientation detection or correction of the image
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
Technology is described for referencing anatomical structure using an AR headset. The method can include registering a marker located on a distal anatomical structure. The marker may have a pose that includes a position and orientation in a 3D coordinate system of the AR headset. Another operation may be identifying a proximal anatomical structure to which the distal anatomical structure is connected. The marker may be registered at a second pose having a second position and second orientation. A joint pivot axis and angle may be determined with respect to the proximal anatomical structure by comparing the first pose and the second pose. The joint pivot axis may then be displayed, using the AR headset. The technology may also use a pointer device to automatically find an anatomic path and anatomic length.
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06V 10/62 - Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extractionPattern tracking
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/50 - Supports for surgical instruments, e.g. articulated arms
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
3.
Augmenting Real-Time Views of a Patient with Three-Dimensional Data
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
G16H 20/40 - ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 40/63 - ICT specially adapted for the management or administration of healthcare resources or facilitiesICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
An AR headset is described to co-localize an image data set with a body of a person. One method can include identifying optical codes with a contrast medium in a tubing on the body of the person using the AR headset. The image data set can be aligned with the body of the person using the fixed position of the image visible marker with the contrast medium in the tubing with respect to the optical code referenced to a representation of the image visible marker. In one configuration, an image data set can be scaled by comparing a measured size of the image visible marker (e.g., gadolinium tubing) in the captured image data set to the known size of the image visible marker. In addition, a center of an optical code may be identified to more accurately align the image data set with a body of a person.
A technology is described for aligning an image data set with a patient using an augmented reality (AR) headset. A method may include obtaining an image data set representing an anatomical structure of a patient. A two-dimensional (2D) X-ray generated image of at least a portion of the anatomical structure of the patient in the image data set and a visible marker may be obtained. The image data set can be aligned to the X-ray generated image by using data fitting. A location of the visible marker may be defined in the image data set using alignment with the X-ray generated image. The image data set may be aligned with a body of the patient, using the visible marker in the image data set as referenced to the visible marker seen on the patient through the AR headset.
Technology is described that includes a method for aligning an image data set with a computed 3D coordinate of a point representing a feature. The method can include receiving 2D (two dimensional) image data from a first medical imaging device, wherein the 2D image data has a location and graphical mark for a point representing a feature. A 3D coordinate of the point in the 3D coordinate system can be computed using a location and orientation of the first medical imaging device with respect to the AR headset. A 3D image data set obtained from a second medical imaging device and a virtual marker identifying the feature in the 3D image data set may be received. The 3D image data set may be aligned to the patient anatomy by aligning the virtual marker in the 3D image data set to the point in the 2D image data set.
Aligning image data of a patient with actual views of the patient using an optical code affixed to the patient. In some embodiments, a method may include affixing an optical code to a patient, affixing a pattern of markers to the patient, capturing image data of the patient, sensing the optical code affixed to the patient and a position of the optical code in a 3D space, accessing the image data, calculating the position of the pattern of markers in the 3D space, registering the position of the inner layer of the patient in the 3D space by aligning the calculated position of the pattern of markers in the 3D space with the position of the pattern of markers in the image data, and displaying in real-time, in an alternate reality (AR) headset, the inner layer of the patient from the image data projected onto actual views of the patient.
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
G06V 10/24 - Aligning, centring, orientation detection or correction of the image
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
8.
Alignment of Medical Images in Augmented Reality Displays
An AR headset is described to co-localize an image data set with a body of a person. The image data set of a portion of the body may be identified. The image data set can include a virtual anatomical marker to identify an anatomical location. A spatial location may be identified using a physical pointer object in a field of view of the AR headset. A spatial location of a visible anatomical feature as identified using the physical pointer object may be recorded. The registration may be triggered based on an input that the physical pointer object is located at a visible anatomical feature of the body of the person. Further, the image data set may be aligned with the body of the person using the spatial location recorded for the visible anatomical feature as referenced to the virtual anatomical marker in the image data set.
Technology is described to enable alignment of an image data set with an anatomy of a person using an augmented reality (AR) headset. The device may include an image visible marker that is detectable by a medical imaging device and an optical marker. A common connector can removably join the image visible marker to the optical marker and define a fixed distance between the image visible marker and the optical marker.
Technology is described to enable alignment of an image data set with an anatomy of a person using an augmented reality (AR) headset. The device may include an image visible marker that is detectable by a medical imaging device and an optical marker. A common connector can removably join the image visible marker to the optical marker and define a fixed distance between the image visible marker and the optical marker.
A technology is described for using a medical implement or a fluoroscopic image with reference to an image data set and a body of a person. A method may include detecting visual image data of a body of a patient and a medical implement. The optical codes on the body of the patient and on the medical implement may be identified. One operation is aligning the image data set with the body of the person using one or more optical codes on the body of the person and the fixed position of an image visible marker with respect to the optical code. A position of the medical implement with respect to the body of the person may be determined using one or more optical codes on the medical implement and the body of the person to reference the medical implement to the image data set and the body of the person.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04N 21/4405 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
12.
3D SPATIAL MAPPING IN A 3D COORDINATE SYSTEM OF AN AR HEADSET USING 2D IMAGES
Technology is described for generating 3D spatial mappings. One operation may be receiving a 2D image of patient anatomy from a medical imaging device. Another operation may be identifying a placement of an imaging device in a 3D coordinate system of an augmented reality (AR) headset when capturing the 2D image. A graphical mark may be received from a user for a point within a body of a person in the 2D image. A 3D coordinate of the point in the 3D coordinate system may be computed using the position and orientation of the imaging device with respect to the AR headset and a depth of the point, in order to generate a 3D spatial mapping.
Technology is described for generating 3D spatial mappings. One operation may be receiving a 2D image of patient anatomy from a medical imaging device. Another operation may be identifying a placement of an imaging device in a 3D coordinate system of an augmented reality (AR) headset when capturing the 2D image. A graphical mark may be received from a user for a point within a body of a person in the 2D image. A 3D coordinate of the point in the 3D coordinate system may be computed using the position and orientation of the imaging device with respect to the AR headset and a depth of the point, in order to generate a 3D spatial mapping.
A technology is described for aligning an image data set with a patient using an augmented reality (AR) headset. A method may include obtaining an image data set representing an anatomical structure of a patient. A two-dimensional (2D) X-ray generated image of at least a portion of the anatomical structure of the patient in the image data set and a visible marker may be obtained. The image data set can be aligned to the X-ray generated image by using data fitting. A location of the visible marker may be defined in the image data set using alignment with the X-ray generated image. The image data set may be aligned with a body of the patient, using the visible marker in the image data set as referenced to the visible marker seen on the patient through the AR headset.
A technology is described for aligning an image data set with a patient using an augmented reality (AR) headset. A method may include obtaining an image data set representing an anatomical structure of a patient. A two-dimensional (2D) X-ray generated image of at least a portion of the anatomical structure of the patient in the image data set and a visible marker may be obtained. The image data set can be aligned to the X-ray generated image by using data fitting. A location of the visible marker may be defined in the image data set using alignment with the X-ray generated image. The image data set may be aligned with a body of the patient, using the visible marker in the image data set as referenced to the visible marker seen on the patient through the AR headset.
A technology is described for using an augmented reality (AR) headset to co-localize an image data set with a body of a person. A method may include registering one or more initial visible markers attached to the body of the person, and the optical code may be located in a fixed position relative to the image visible marker. The coordinate system of the one or more initial visible markers may be transferred to the one or more additional visible markers. The use of the one or more additional visible markers may be emphasized to align of the image data set with the body of the person.
A technology is described for using an augmented reality (AR) headset to co- localize an image data set with a body of a person. A method may include registering one or more initial visible markers attached to the body of the person, and the optical code may be located in a fixed position relative to the image visible marker. The coordinate system of the one or more initial visible markers may be transferred to the one or more additional visible markers. The use of the one or more additional visible markers may be emphasized to align of the image data set with the body of the person.
Technology is described to adjust for eye location variations of a user when using an augmented reality (AR) headset. The method can include registering a position and size of an optical code using a visual camera and the AR headset. An alignment marker may be projected through the AR headset to be aligned with the optical code for a right eye and left eye of a user of the AR headset. Right eye and left eye adjustments to the alignment marker may be received independently to align the alignment marker with a portion of the optical code as viewed by either the right eye or left eye of the user. The left eye adjustments and right eye adjustments may be applied to virtual images displayed through the AR headset in order to improve the accuracy of alignment between objects in the physical view and the virtual images displayed using the AR headset.
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
G06T 19/20 - Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
G16H 20/40 - ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 40/63 - ICT specially adapted for the management or administration of healthcare resources or facilitiesICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
A technology is described for using a medical implement or a fluoroscopic image with reference to an image data set and a body of a person. A method may include detecting visual image data of a body of a patient and a medical implement. The optical codes on the body of the patient and on the medical implement may be identified. One operation is aligning the image data set with the body of the person using one or more optical codes on the body of the person and the fixed position of an image visible marker with respect to the optical code. A position of the medical implement with respect to the body of the person may be determined using one or more optical codes on the medical implement and the body of the person to reference the medical implement to the image data set and the body of the person.
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computerOutput arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06T 19/00 - Manipulating 3D models or images for computer graphics
H04N 21/4405 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
21.
ALIGNMENT OF MEDICAL IMAGES IN AUGMENTED REALITY DISPLAYS
An AR headset is described to co-localize an image data set with a body of a person. The image data set of a portion of the body may be identified. The image data set can include a virtual anatomical marker to identify an anatomical location. A spatial location may be identified using a physical pointer object in a field of view of the AR headset. A spatial location of a visible anatomical feature as identified using the physical pointer object may be recorded. The registration may be triggered based on an input that the physical pointer object is located at a visible anatomical feature of the body of the person. Further, the image data set may be aligned with the body of the person using the spatial location recorded for the visible anatomical feature as referenced to the virtual anatomical marker in the image data set.
An AR headset is described to co-localize an image data set with a body of a person. The image data set of a portion of the body may be identified. The image data set can include a virtual anatomical marker to identify an anatomical location. A spatial location may be identified using a physical pointer object in a field of view of the AR headset. A spatial location of a visible anatomical feature as identified using the physical pointer object may be recorded. The registration may be triggered based on an input that the physical pointer object is located at a visible anatomical feature of the body of the person. Further, the image data set may be aligned with the body of the person using the spatial location recorded for the visible anatomical feature as referenced to the virtual anatomical marker in the image data set.
Technology is described for augmenting medical imaging for use in a medical procedure. The method can include the operation of receiving an image of patient anatomy captured by a visual image camera during the medical procedure. An acquired medical image associated with the patient anatomy can then be retrieved. Another operation can be associating the acquired medical image to the patient anatomy. An augmentation tag associated with a location in one layer of the acquired medical image can be retrieved. A further operation can be projecting the acquired medical image and the augmentation tag using an augmented reality headset to form a single graphical view as an overlay to the patient anatomy in either 2D, 3D or holographic form.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
A61B 5/06 - Devices, other than using radiation, for detecting or locating foreign bodies
A61B 5/107 - Measuring physical dimensions, e.g. size of the entire body or parts thereof
A61B 90/96 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
A61B 90/98 - Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
A61B 1/04 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor combined with photographic or television appliances
A61B 1/00 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor
A61B 90/90 - Identification means for patients or instruments, e.g. tags
A61B 90/94 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 50/50 - ICT specially adapted for medical diagnosis, medical simulation or medical data miningICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
24.
Alignment of medical images in augmented reality displays
An AR headset is described to co-localize an image data set with a body of a person. One method can include identifying optical codes with a contrast medium in a tubing on the body of the person using the AR headset. The image data set can be aligned with the body of the person using the fixed position of the image visible marker with the contrast medium in the tubing with respect to the optical code referenced to a representation of the image visible marker. In one configuration, an image data set can be scaled by comparing a measured size of the image visible marker (e.g., gadolinium tubing) in the captured image data set to the known size of the image visible marker. In addition, a center of an optical code may be identified to more accurately align the image data set with a body of a person.
Technology is described for augmenting medical imaging for use in a medical procedure. The method can include the operation of receiving an image of patient anatomy captured by a visual image camera during the medical procedure. An acquired medical image associated with the patient anatomy can then be retrieved. Another operation can be associating the acquired medical image to the patient anatomy. An augmentation tag associated with a location in one layer of the acquired medical image can be retrieved. A further operation can be projecting the acquired medical image and the augmentation tag using an augmented reality headset to form a single graphical view as an overlay to the patient anatomy in either 2D, 3D or holographic form.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
A61B 5/06 - Devices, other than using radiation, for detecting or locating foreign bodies
A61B 5/107 - Measuring physical dimensions, e.g. size of the entire body or parts thereof
A61B 90/96 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
A61B 90/98 - Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
A61B 1/04 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor combined with photographic or television appliances
A61B 1/00 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor
A61B 90/90 - Identification means for patients or instruments, e.g. tags
A61B 90/94 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 50/50 - ICT specially adapted for medical diagnosis, medical simulation or medical data miningICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
A61B 90/50 - Supports for surgical instruments, e.g. articulated arms
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
A61B 17/00 - Surgical instruments, devices or methods
26.
Augmenting real-time views of a patient with three-dimensional data
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
G16H 20/40 - ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 40/63 - ICT specially adapted for the management or administration of healthcare resources or facilitiesICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
27.
Alignment of medical images in augmented reality displays
An AR headset is described to co-localize an image data set with a body of a person. One method can include identifying optical codes with a contrast medium in a tubing on the body of the person using the AR headset. The image data set can be aligned with the body of the person using the fixed position of the image visible marker with the contrast medium in the tubing with respect to the optical code referenced to a representation of the image visible marker. In one configuration, an image data set can be scaled by comparing a measured size of the image visible marker (e.g., gadolinium tubing) in the captured image data set to the known size of the image visible marker. In addition, a center of an optical code may be identified to more accurately align the image data set with a body of a person.
An AR headset is described to co-localize an image data set with a body of a person. One method can include identifying optical codes with a contrast medium in a tubing on the body of the person using the AR headset. The image data set can be aligned with the body of the person using the fixed position of the image visible marker with the contrast medium in the tubing with respect to the optical code referenced to a representation of the image visible marker. In one configuration, an image data set can be scaled by comparing a measured size of the image visible marker (e.g., gadolinium tubing) in the captured image data set to the known size of the image visible marker. In addition, a center of an optical code may be identified to more accurately align the image data set with a body of a person.
Aligning image data of a patient with actual views of the patient using an optical code affixed to the patient. In some embodiments, a method may include affixing an optical code to a patient, affixing a pattern of markers to the patient, capturing image data of the patient, sensing the optical code affixed to the patient and a position of the optical code in a 3D space, accessing the image data, calculating the position of the pattern of markers in the 3D space, registering the position of the inner layer of the patient in the 3D space by aligning the calculated position of the pattern of markers in the 3D space with the position of the pattern of markers in the image data, and displaying in real-time, in an alternate reality (AR) headset, the inner layer of the patient from the image data projected onto actual views of the patient.
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
G06V 10/24 - Aligning, centring, orientation detection or correction of the image
G06V 20/20 - ScenesScene-specific elements in augmented reality scenes
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
30.
SECURE ACCESS TO STORED DATA FILES USING TOKENS ENCODED IN OPTICAL CODES
Technology is described for storing and reading a data file using a token encoded in a printed optical code. The method can include receiving the data file over a computer network from a client device. The data file can be sent to be stored in a data store of virtualized data storage accessible via the internet. A token may be received from the data store and the token can be used to access the data file. The data file is returned from the data store when the token is later received from a client device. The token is encoded into an optical code. The optical code can be sent to the client device, where the client device has access to a printer to print the optical code with the token encoded. Any electronic copy of the token can be destroyed.
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
31.
Secure Access to Stored Data Files Using Tokens Encoded in Optical Codes
Technology is described for storing and reading a data file using a token encoded in a printed optical code. The method can include receiving the data file over a computer network from a client device. The data file can be sent to be stored in a data store of virtualized data storage accessible via the internet. A token may be received from the data store and the token can be used to access the data file. The data file is returned from the data store when the token is later received from a client device. The token is encoded into an optical code. The optical code can be sent to the client device, where the client device has access to a printer to print the optical code with the token encoded. Any electronic copy of the token can be destroyed.
G16H 10/60 - ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
G06F 16/955 - Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
32.
USING OPTICAL CODES WITH AUGMENTED REALITY DISPLAYS
A technology is described for using a medical implement or a fluoroscopic image with reference to an image data set and a body of a person. A method may include detecting visual image data of a body of a patient and a medical implement. The optical codes on the body of the patient and on the medical implement may be identified. One operation is aligning the image data set with the body of the person using one or more optical codes on the body of the person and the fixed position of an image visible marker with respect to the optical code. A position of the medical implement with respect to the body of the person may be determined using one or more optical codes on the medical implement and the body of the person to reference the medical implement to the image data set and the body of the person.
A technology is described for using a medical implement or a fluoroscopic image with reference to an image data set and a body of a person. A method may include detecting visual image data of a body of a patient and a medical implement. The optical codes on the body of the patient and on the medical implement may be identified. One operation is aligning the image data set with the body of the person using one or more optical codes on the body of the person and the fixed position of an image visible marker with respect to the optical code. A position of the medical implement with respect to the body of the person may be determined using one or more optical codes on the medical implement and the body of the person to reference the medical implement to the image data set and the body of the person.
G06F 3/00 - Input arrangements for transferring data to be processed into a form capable of being handled by the computerOutput arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
H04N 21/4405 - Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
G06T 19/00 - Manipulating 3D models or images for computer graphics
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
34.
Augmenting real-time views of a patient with three-dimensional data
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
G16H 20/40 - ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 40/63 - ICT specially adapted for the management or administration of healthcare resources or facilitiesICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
35.
Augmented reality viewing and tagging for medical procedures
Technology is described for augmenting medical imaging for use in a medical procedure. The method can include the operation of receiving an image of patient anatomy captured by a visual image camera during the medical procedure. An acquired medical image associated with the patient anatomy can then be retrieved. Another operation can be associating the acquired medical image to the patient anatomy. An augmentation tag associated with a location in one layer of the acquired medical image can be retrieved. A further operation can be projecting the acquired medical image and the augmentation tag using an augmented reality headset to form a single graphical view as an overlay to the patient anatomy in either 2D, 3D or holographic form.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
A61B 5/06 - Devices, other than using radiation, for detecting or locating foreign bodies
A61B 5/107 - Measuring physical dimensions, e.g. size of the entire body or parts thereof
A61B 90/96 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
A61B 90/98 - Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
A61B 1/04 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor combined with photographic or television appliances
A61B 1/00 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor
A61B 90/90 - Identification means for patients or instruments, e.g. tags
A61B 90/94 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
G16H 50/50 - ICT specially adapted for medical diagnosis, medical simulation or medical data miningICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
A61B 90/50 - Supports for surgical instruments, e.g. articulated arms
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
A61B 17/00 - Surgical instruments, devices or methods
36.
ALIGNING IMAGE DATA OF A PATIENT WITH ACTUAL VIEWS OF THE PATIENT USING AN OPTICAL CODE AFFIXED TO THE PATIENT
Aligning image data of a patient with actual views of the patient using an optical code affixed to the patient. In some embodiments, a method may include affixing an optical code to a patient, affixing a pattern of markers to the patient, capturing image data of the patient, sensing the optical code affixed to the patient and a position of the optical code in a 3D space, accessing the image data, calculating the position of the pattern of markers in the 3D space, registering the position of the inner layer of the patient in the 3D space by aligning the calculated position of the pattern of markers in the 3D space with the position of the pattern of markers in the image data, and displaying in real- time, in an alternate reality (AR) headset, the inner layer of the patient from the image data projected onto actual views of the patient.
Aligning image data of a patient with actual views of the patient using an optical code affixed to the patient. In some embodiments, a method may include affixing an optical code to a patient, affixing a pattern of markers to the patient, capturing image data of the patient, sensing the optical code affixed to the patient and a position of the optical code in a 3D space, accessing the image data, calculating the position of the pattern of markers in the 3D space, registering the position of the inner layer of the patient in the 3D space by aligning the calculated position of the pattern of markers in the 3D space with the position of the pattern of markers in the image data, and displaying in real-time, in an alternate reality (AR) headset, the inner layer of the patient from the image data projected onto actual views of the patient.
G16H 30/20 - ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
G06K 7/14 - Methods or arrangements for sensing record carriers by electromagnetic radiation, e.g. optical sensingMethods or arrangements for sensing record carriers by corpuscular radiation using light without selection of wavelength, e.g. sensing reflected white light
G06K 19/06 - Record carriers for use with machines and with at least a part designed to carry digital markings characterised by the kind of the digital marking, e.g. shape, nature, code
38.
Augmenting real-time views of a patient with three-dimensional data
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
G06F 19/00 - Digital computing or data processing equipment or methods, specially adapted for specific applications (specially adapted for specific functions G06F 17/00;data processing systems or methods specially adapted for administrative, commercial, financial, managerial, supervisory or forecasting purposes G06Q;healthcare informatics G16H)
Technology is described for incorporating statistical strength from neighboring voxels in an fMRI image. The method can include the operation of capturing an fMRI image of a human organ and the fMRI image includes statistical values for voxels in order to detect changes associated with blood flow representing organ activity. The statistics of the fMRI image can be upsampled to a larger coordinate size. Another operation can be resampling the statistical values of the fMRI image to improve the statistical strength for target voxels in the fMRI by identifying strong statistical values in neighboring voxels in a defined neighborhood of the target voxels and improving statistical values of the target voxels using the strong statistical values. The statistics of the fMRI image can integrated into a region of interest identified for the human organ to improve data values of the target voxels while retaining regional boundaries.
Technology is described for augmenting medical imaging for use in a medical procedure. The method can include the operation of receiving an image of patient anatomy captured by a visual image camera during the medical procedure. An acquired medical image associated with the patient anatomy can then be retrieved. Another operation can be associating the acquired medical image to the patient anatomy. An augmentation tag associated with a location in one layer of the acquired medical image can be retrieved. A further operation can be projecting the acquired medical image and the augmentation tag using an augmented reality headset to form a single graphical view as an overlay to the patient anatomy in either 2D, 3D or holographic form.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/96 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
A61B 1/04 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor combined with photographic or television appliances
A61B 5/06 - Devices, other than using radiation, for detecting or locating foreign bodies
A61B 5/107 - Measuring physical dimensions, e.g. size of the entire body or parts thereof
A61B 90/98 - Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06T 7/73 - Determining position or orientation of objects or cameras using feature-based methods
G06F 19/00 - Digital computing or data processing equipment or methods, specially adapted for specific applications (specially adapted for specific functions G06F 17/00;data processing systems or methods specially adapted for administrative, commercial, financial, managerial, supervisory or forecasting purposes G06Q;healthcare informatics G16H)
Technology is described for augmenting medical imaging for use in a medical procedure. The method can include the operation of receiving an image of patient anatomy captured by a visual image camera during the medical procedure. An acquired medical image associated with the patient anatomy can then be retrieved. Another operation can be associating the acquired medical image to the patient anatomy. An augmentation tag associated with a location in one layer of the acquired medical image can be retrieved. A further operation can be projecting the acquired medical image and the augmentation tag using an augmented reality headset to form a single graphical view as an overlay to the patient anatomy in either 2D, 3D or holographic form.
G09G 5/00 - Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/96 - Identification means for patients or instruments, e.g. tags coded with symbols, e.g. text using barcodes
A61B 1/04 - Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopesIlluminating arrangements therefor combined with photographic or television appliances
A61B 5/06 - Devices, other than using radiation, for detecting or locating foreign bodies
A61B 5/107 - Measuring physical dimensions, e.g. size of the entire body or parts thereof
A61B 90/98 - Identification means for patients or instruments, e.g. tags using electromagnetic means, e.g. transponders
G06T 19/00 - Manipulating 3D models or images for computer graphics
G06F 19/00 - Digital computing or data processing equipment or methods, specially adapted for specific applications (specially adapted for specific functions G06F 17/00;data processing systems or methods specially adapted for administrative, commercial, financial, managerial, supervisory or forecasting purposes G06Q;healthcare informatics G16H)
Augmenting real-time views of a patient with three-dimensional (3D) data. In one embodiment, a method may include identifying 3D data for a patient with the 3D data including an outer layer and multiple inner layers, determining virtual morphometric measurements of the outer layer from the 3D data, registering a real-time position of the outer layer of the patient in a 3D space, determining real-time morphometric measurements of the outer layer of the patient, automatically registering the position of the outer layer from the 3D data to align with the registered real-time position of the outer layer of the patient in the 3D space using the virtual morphometric measurements and using the real-time morphometric measurements, and displaying, in an augmented reality (AR) headset, one of the inner layers from the 3D data projected onto real-time views of the outer layer of the patient.
A method for monitoring productivity may include monitoring performance of a medical procedure at a local device. Medical procedure data may be received at the local device from a management device. The medical procedure data may include relative value units (RVUs) associated with the medical procedure. Medical professional productivity data related to the medical procedure may be provided for display at the local device based on the monitoring of the performance of the medical procedure and the medical procedure data. A procedure completion notification may be transmitted from the local device to the management device when the medical procedure is completed.
G06F 15/02 - Digital computers in generalData processing equipment in general manually operated with input through keyboard and computation using a built-in program, e.g. pocket calculators
G06Q 50/00 - Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
G06Q 10/06 - Resources, workflows, human or project managementEnterprise or organisation planningEnterprise or organisation modelling
A method of creating a panoramic peripheral image may include identifying a virtual observer at a position in a modeled scene. Peripheral images of the modeled scene may be obtained, including views at multiple angles from the virtual observer. The peripheral images may be arranged to depict a panoramic peripheral view of the modeled scene.
G06F 19/00 - Digital computing or data processing equipment or methods, specially adapted for specific applications (specially adapted for specific functions G06F 17/00;data processing systems or methods specially adapted for administrative, commercial, financial, managerial, supervisory or forecasting purposes G06Q;healthcare informatics G16H)
G06T 17/20 - Wire-frame description, e.g. polygonalisation or tessellation
patents
