A computing system and method are provided for image navigation using on-demand deep learning based segmentation. An example system may comprise an exoscope configured to capture image data from a field of view; an image segmentation module; an intent recognition module to capture a user's intent; and one or more robotic arms configured to move the exoscope. The system may receive, via the exoscope, image data relating to an image or a video stream of a surgical site; generate, via the image segmentation module, an augmented image comprising a plurality of labeled regions overlaying the surgical site; receive, via the intent recognition module, a voice command selecting a labeled region of the plurality of labeled regions; and cause, via the one or more robotic arms, a movement of the exoscope so that the selected labeled region is within the field of view of the exoscope after the movement.
A stereoscopic camera with fluorescence strobing based visualization is disclosed herein. In an example, a stereoscopic camera is configured to provide a visible light mode for a first specified number of frames over a cycle by causing visible light reflected from a surgical site to be provided to left and right image sensors by activating a visible light source. The stereoscopic camera is also configured to provide a fluorescence mode for a second specified number of frames over the cycle by causing fluorescence emission light from the surgical site to be provided to the left and right image sensors by activating a near-ultraviolet light source. The stereoscopic camera switches between the visible light mode and the fluorescence mode based on the first and second specified number of frames. A processor superimposes image data corresponding to the fluorescence mode on subsequently received image data corresponding to the visible light mode.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/30 - Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
H04N 23/56 - Cameras or camera modules comprising electronic image sensorsControl thereof provided with illuminating means
H04N 23/667 - Camera operation mode switching, e.g. between still and video, sport and normal or high and low resolution modes
Deep learning based systems and methods are disclosed for integrating, displaying, and drawing inferences from surgical information from disparate medical devices. An example system comprises: a plurality of medical devices, each comprising an interface for outputting image, video, and/or audio data; a sensor associated with each medical device configured to capture raw image, video, and/or audio data from the respective interface; a plurality of data gathering modules configured to transmit processed image, video, and/or audio data from the plurality of medical devices to a computing device; the computing device comprising: a processor; and memory storing computer-executable instructions that, when executed by the processor, causes the system to: receive, in real-time, the processed image, video, and/or audio data from each of the plurality of medical devices; and apply a deep learning model to the processed image, video, and/or audio data to generate an output outcome.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
G16H 10/60 - ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
G16H 30/40 - ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
A61B 17/00 - Surgical instruments, devices or methods
A61B 5/00 - Measuring for diagnostic purposes Identification of persons
5.
MICROPHONE DIRECTIONALITY CONTROL BASED ON SURGEON'S COMMAND
New and innovative systems and methods are described for providing microphone directionality based on a surgeon's command, for use in surgical environmnents. An example method may include: receiving, via a respective sensor for each of one or more rotating elements of each of one or more robotic arms connecting a digital surgical microscope to the computing device, an angle information for each rotating element; determining, based on the angle information for each rotating element, a joint angle information for the digital surgical microscope; determining, based on the joint angle information, a location of a head of the digital surgical microscope respective to a microphone device; and activating, based on the location, a first channel of a plurality of channels of the microphone device.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
Systems and methods for registration degradation correction for surgical procedures are disclosed. An example system includes a processor and a surgical camera configured to record images of a patient. The processor is configured to perform an initial patient registration that registers a patient volume space of virtual positional data points to physical positional points of at least a portion of the patient. The processor also identifies or receives an indication of an identification of a natural patient mark using recorded images of the patient and records a virtual mark in the patient volume space in response to a received activation action based on the identification of the natural patient mark. The processor then causes the patient volume space of virtual positional data points and the recorded virtual mark to be displayed in an overlaid manner over the recorded images on a single display.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
New and innovative systems and methods for an integrated surgical navigation and visualization are disclosed. An example system comprises: a single cart providing mobility; a stereoscopic digital surgical microscope; one or more computing devices (e.g., including a single computing device) housing and jointly executing a surgical navigation module and a surgical visualization module, and powered by a single power connection, thus reducing operating room footprint; a single unified display; a processor; and memory. In one embodiment, the system may control a position of a stereoscopic digital surgical microscope with a given reference; provide navigation of a surgical site responsive to user input; provide visualization of the surgical site via a single unified display; and synchronize, in real time, the visualization by integrating navigation information and the visualization of the surgical via the single unified display.
Systems and methods are disclosed for controlling a surgical visualization system using a hands-free (e.g., mouth, nose. and breath actuated) controller. An example system includes a microscope camera associated with the surgical system; a controller of the microscope camera; a processor; and a memory storing instructions for the processor. The controller may be separate from the microscope camera, and may comprise one or more joysticks. In some aspects, the controller may further include a pressure detector configured to detect pressure within a tube of a joystick. Also or alternatively, the controller may include a keyed proximity sensor to activate the controller when a surgeon is present. The processor may be configured to: receive a movement input based on a movement of the joystick along a Cartesian direction; and cause a corresponding movement of the microscope camera along the Cartesian direction.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
Systems and methods are disclosed for an automated touchless registration of patient anatomy for surgical navigation systems. An example method includes: acquiring a plurality of images of the patient; determining relevant objects from a plurality of poses; and processing the received images through a photogrammetry module. The photogrammetry module may provide camera calibration and facilitate camera registration. As such, manual movement or robotic movement capabilities of the camera head, and digital visualization capabilities of a digital surgical microscope disclosed herein can be extended to allow near-automated or fully automated touchless registration for traditional surgical navigation.
The present disclosure provides new and innovative systems and apparatuses for a surgical registration probe that provides improved detection by a surgical navigation system (e.g., localizer). An example apparatus includes a surgical registration probe; a localizer configured to track the surgical registration probe; and a surgical visualization system (e.g., localizer) configured to display a surgical site responsive to the tracking. The surgical registration probe may include: a marker shaft with at least one marker or fiducial; wherein the marker or fiducial causes the localizer to detect the surgical registration probe; and a probe shaft having at least three sections that are bent at a defined angle with respect to each other. In some embodiments, the apparatus may further comprise a calibration plate for securing the surgical registration probe.
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 17/00 - Surgical instruments, devices or methods
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor, sold as a unit; Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor which includes integrated surgical navigation technology, sold as a unit
Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor, sold as a unit; Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor which includes integrated surgical navigation technology, sold as a unit
13.
SYSTEM, METHOD, AND APPARATUS FOR TRACKING A TOOL VIA A DIGITAL SURGICAL MICROSCOPE
The present disclosure relates generally to a system, method, and apparatus for tracking a tool via a digital surgical microscope. Cameras on the digital surgical microscope may capture a scene view of a medical procedure in real time, and present the scene view to the surgeon in a digitized video stream with minimal interference from the surgeon. The digital surgical microscope may process image data from each scene view in real time and use computer vision and machine learning models (e.g., neural networks) to detect and track one or more tools used over the course of the medical procedure in real-time. As the digital surgical microscope detects and tracks the tools, and responds accordingly, the surgeon can thus indirectly control, using the tools already in the surgeon's hands, various parameters of the digital surgical microscope, including the position and orientation of the robotic-arm-mounted digital surgical microscope.
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
G06V 10/25 - Determination of region of interest [ROI] or a volume of interest [VOI]
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersectionsConnectivity analysis, e.g. of connected components
Medical imaging device for use in medical and surgical procedures and evaluations; Digital microscope for medical and surgical procedures, comprised of a robotic arm, 3D camera, and embedded software for processing and recording 2D and 3D images, for use with a 3D monitor; Digital surgical visualization platform (device/hardware) for medical and surgical procedures with a robotic arm, 3D camera, and embedded software for processing and recording 2D and 3D images, for use with a 3D monitor
Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor, sold as a unit; Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor which includes integrated surgical navigation technology, sold as a unit
Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor, sold as a unit; Digital surgical visualization platform in the nature of a medical imaging apparatus comprised of a robotic enabled 3D camera and also including embedded recorded computer software for conducting medical imaging on a 3D monitor which includes integrated surgical navigation technology, sold as a unit
17.
Registration degradation correction for surgical navigation procedures
New and innovative systems and methods for registration degradation correction for surgical procedures are disclosed. An example system includes a surgical marking pen including a first trackable target and a registration plate including a second trackable target. The system also includes a navigation camera and a processor configured to perform a pen registration that determines a transformation between a tip of the surgical marking pen and the first trackable target when the tip of the surgical marking pen is placed on the registration plate. The pen registration enables the processor to record virtual marks at locations of the pen tip that correspond to physical marks drawn by the pen. Locations of the virtual marks are later compared to images of the physical marks to correct any registration degradation by moving a surgical camera or robotic arm connected to the surgical camera.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
New and innovative systems and methods for auto-navigation in an integrated surgical navigation and visualization system are disclosed. An example system comprises: a single cart providing motility; a stereoscopic digital surgical microscope comprising a surgical visualization camera and a localizer; one or more computing devices (e.g., a single computing device powered by a single power connection) housing and jointly executing a surgical navigation module and a surgical visualization module, wherein the localizer is associated with the surgical navigation module, and wherein the surgical visualization camera is associated with the surgical visualization module; a single unified display; a processor; and memory. The system may generate a transformation of patient data associated with a patient to the surgical visualization camera; calibrate the surgical visualization camera and the localizer; provide visualization of the surgical site via the single unified display; and provide navigation of the surgical site responsive to user input.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
New and innovative systems and methods are described for providing microphone directionality based on a surgeon's command, for use in surgical environments. An example method may include: receiving, via a respective sensor for each of one or more rotating elements of each of one or more robotic arms connecting a digital surgical microscope to the computing device, an angle information for each rotating element; determining, based on the angle information for each rotating element, a joint angle information for the digital surgical microscope; determining, based on the joint angle information, a location of a head of the digital surgical microscope respective to a microphone device; and activating, based on the location, a first channel of a plurality of channels of the microphone device.
G10L 15/20 - Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise or of stress induced speech
G10L 21/00 - Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
G10L 21/06 - Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
H04R 1/32 - Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
20.
SYSTEM, METHOD, AND APPARATUS FOR HAND-CENTRIC CONTROLLER FOR THE ROBOTIC DIGITAL SURGICAL MICROSCOPE
The present disclosure relates generally a hand-centric controller that provides a user (e.g., surgeon) with the ability to control a number of microscope movement controls, non-movement microscope controls, image and color controls, media controls, and hyperspectral controls without having to reach beyond the space surrounding the surgical tool being used or the space surrounding the surgeon's hands. In some embodiments, the hand-centric controller is a limited button (e.g., one, two, three buttons) controller. In other embodiments, the hand-centric controller is an extended hand-centric controller. The hand-centric controller may be configured to provide microscope movement (e.g., x-y axis movement, lock-to-target movement, yaw movement, physical focus movement, and gross general movement), non-movement microscope control (e.g., zoom, focus, autofocus, and white light), image and color controls (e.g., next image and previous image modes), media controls (e.g., snapshot control, stop and start recording modes), and hyperspectral controls (e.g., DIR 800 on/off, light control, and playback, and DUV 400 on/off and light control).
A61B 34/00 - Computer-aided surgeryManipulators or robots specially adapted for use in surgery
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/20 - Surgical microscopes characterised by non-optical aspects
H04N 23/62 - Control of parameters via user interfaces
H04N 23/66 - Remote control of cameras or camera parts, e.g. by remote control devices
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
21.
COVER CAP AND ASSEMBLY KIT CONSISTING OF COVER CAP AND BASE CAP
The present disclosure relates to a cover cap (2) for a medical component, in particular for a robotic arm, with a circular cylindrical tube section (4) that is closed at its one longitudinal end and is formed open at its other longitudinal end and has an edge (6) terminating the tube section (4) at the open longitudinal end, wherein the edge (6) lies in a plane inclined in relation to a tube section longitudinal axis, and at least three catch tabs (8, 10, 12) arranged at a distance from one another over the circumference of the tube section (4) and projecting radially inward from the edge (6) for attaching the cover cap (2) by clip-on connection. The present disclosure also relates to an assembly kit having such a cover cap (2) and a base cap (40) over which the cover cap (2) can be seated.
A localization target for a digital surgical stereoscope is disclosed herein. In an example, the localization target includes a shell apparatus for a surgical imaging camera. The apparatus includes a top surface integrally formed with a front surface and two opposing side surfaces defining empty space therebetween. Each of the top surface, the side surfaces, and the front surface includes at least three tracking features. The apparatus also includes at least six kinematic constraints located on an internally facing side of at least one of the top surface, the side surfaces, or the front surface. The apparatus further includes a connector that is positioned within an area defined by the at least six kinematic constraints.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A stereoscopic camera with fluorescence strobing based visualization is disclosed herein. In an example, a stereoscopic camera is configured to provide a visible light mode for a first specified number of frames over a cycle by causing visible light reflected from a surgical site to be provided to left and right image sensors by activating a visible light source. The stereoscopic camera is also configured to provide a fluorescence mode for a second specified number of frames over the cycle by causing fluorescence emission light from the surgical site to be provided to the left and right image sensors by activating a near-ultraviolet light source. The stereoscopic camera switches between the visible light mode and the fluorescence mode based on the first and second specified number of frames. A processor superimposes image data corresponding to the fluorescence mode on subsequently received image data corresponding to the visible light mode.
H04N 23/56 - Cameras or camera modules comprising electronic image sensorsControl thereof provided with illuminating means
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
A61B 90/30 - Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
H04N 13/239 - Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
H04N 23/667 - Camera operation mode switching, e.g. between still and video, sport and normal or high and low resolution modes
Video overlay synchronization for a digital surgical stereoscope is disclosed herein. In an example, a system provides synchronization by detecting how an image changes between frames. The image change corresponds to detecting the regular and predictable movement of human tissue, such as pulsing of a blood vessel. Peaks or maximum extents of tissue movement is tracked overtime to determine an estimation of the regular periodic movement. This periodic movement is used to align a short recording of images corresponding to a fluorescence imaging mode with a live or near-live recording of images in a visible light mode. The frame rate of fluorescence image data is adjusted to ensure close or almost exact alignment of tissue position shown in both the fluorescence image data and visible light image data. The system accordingly provides a fluorescence image overlay on visible light images for tissue that has regular movement patterns.
Systems and methods are disclosed for controlling a surgical visualization system using a hands-free (e.g., mouth, nose, and breath actuated) controller. An example system includes a microscope camera associated with the surgical system; a controller of the microscope camera; a processor; and a memory storing instructions for the processor. The controller may be separate from the microscope camera, and may comprise one or more joysticks. In some aspects, the controller may further include a pressure detector configured to detect pressure within a tube of a joystick. Also or alternatively, the controller may include a keyed proximity sensor to activate the controller when a surgeon is present. The processor may be configured to: receive a movement input based on a movement of the joystick along a Cartesian direction; and cause a corresponding movement of the microscope camera along the Cartesian direction.
New and innovative systems and methods for calibrating and correcting sensors associated with a collaborative robot are disclosed. An example system comprises: at least one robotic arm; a sensor affixed to a location on the robotic arm, wherein the sensor measures force and torque across six degrees of freedom (6DOF); a processor; and memory. The system may receive, from the sensor, sensor input in real-time that indicate a measured force or torque. The system may generate, in real-time, sensor corrections that correspond to offset, linear, and non-linear deviations of the measured force in each sensor axis. The sensor corrections may correspond to offset, linear, and non-linear cross-coupling of the measured force between two or more sensor axes. The sensor corrections may be determined by applying offset, linear, and non nonlinear corrections to each degree of freedom (DOF) from every other DOF.
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
New and innovative systems and methods for an integrated surgical navigation and visualization are disclosed. An example system comprises: a single cart providing mobility; a stereoscopic digital surgical microscope; one or more computing devices (e.g., including a single computing device) housing and jointly executing a surgical navigation module and a surgical visualization module, and powered by a single power connection, thus reducing operating room footprint; a single unified display; a processor; and memory. In one embodiment, the system may control a position of a stereoscopic digital surgical microscope with a given reference; provide navigation of a surgical site responsive to user input; provide visualization of the surgical site via a single unified display; and synchronize, in real time, the visualization by integrating navigation information and the visualization of the surgical via the single unified display.
A61B 17/00 - Surgical instruments, devices or methods
A61B 34/10 - Computer-aided planning, simulation or modelling of surgical operations
A61B 34/20 - Surgical navigation systemsDevices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
28.
SYSTEM, METHOD, AND APPARATUS FOR HAND-CENTRIC CONTROLLER FOR THE ROBOTIC DIGITAL SURGICAL MICROSCOPE
The present disclosure relates generally a hand-centric controller that provides a user (e.g., surgeon) with the ability to control a number of microscope movement controls, non-movement microscope controls, image and color controls, media controls, and hyperspectral controls without having to reach beyond the space surrounding the surgical tool being used or the space surrounding the surgeon's hands. In some embodiments, the hand-centric controller is a limited button (e.g., one, two, three buttons) controller. In other embodiments, the hand-centric controller is an extended hand-centric controller. The hand-centric controller may be configured to provide microscope movement (e.g., x-y axis movement, lock-to-target movement, yaw movement, physical focus movement, and gross general movement), non-movement microscope control (e.g., zoom, focus, autofocus, and white light), image and color controls (e.g., next image and previous image modes), media controls (e.g., snapshot control, stop and start recording modes), and hyperspectral controls (e.g., DIR 800 on/off, light control, and playback, and DUV 400 on/off and light control).
A61B 90/00 - Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups , e.g. for luxation treatment or for protecting wound edges
29.
Mounting adapter for securing a sterile cover on a microscope, microscope for use with an adapter of this type, and system having a microscope of this type and an adapter of this type
A mounting adapter for detachably attaching a sterile cover to an objective of a microscope includes a passage for inserting the objective. A drape-lens or light-transparent disc is arranged in the passage such that it adopts an oblique angle to the longitudinal axis of the passage. The mounting adapter includes a latching device for an axially holding of the mounting adapter on the objective inserted into the passage. A positioning geometry causes a reproducible target angular positioning or target rotational alignment of the mounting adapter on the objective by its axial attachment to fulfil its positioning function independently and separately from the latching device.
42 - Scientific, technological and industrial services, research and design
Goods & Services
Customizing computer software; Design and development of computer hardware; Design, development, and implementation of software for computer guided microsurgery; Developing and updating computer software; Image processing software design; Installation and maintenance of computer software; Installation of computer software; Software design and development; Software development in the field of medical computer guided mircrosurgery
42 - Scientific, technological and industrial services, research and design
Goods & Services
Customizing computer software; Design and development of computer hardware; Design, development, and implementation of software for computer guided microsurgery; Developing and updating computer software; Image processing software design; Installation and maintenance of computer software; Installation of computer software; Software design and development; Software development in the field of medical computer guided mircrosurgery
