A method of monitoring hierarchical asset groups includes the operations of: receiving a plurality of asset group configuration parameters comprising one or more association specifications, each association specification defining an association between an asset and a corresponding asset group of a plurality of asset groups of an asset hierarchy; receiving definitions of one or more asset group metrics, wherein each asset group metric of the one or more asset group metrics is derived from a search of machine data associated with a corresponding asset group; associating, with each asset group of the plurality of asset groups, a corresponding asset group metric of the one or more asset group metrics; generating one or more alert definitions associated with an asset group of the plurality of asset groups; monitoring, using the one or more asset group metrics, the asset hierarchy; and generating an alert based on the one or more alert definitions.
A computer-implemented method for performing cross regional disaster recovery includes receiving event data at an active deployment of a data intake and query system. The method further includes processing the event data in a plurality of stages of the data intake and query system to convert the event data into searchable buckets of indexed data, wherein files generated at each stage of the processing are uploaded to a first scalable storage module in the active deployment. Further, the method includes replicating the files generated at each stage of the processing to a second scalable storage module in a standby deployment of the data intake and query system. Responsive to an outage at the active deployment, the method includes transitioning control to the standby deployment and recovering the event data at the standby deployment using the files replicated to the second scalable storage module.
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
G06F 16/11 - File system administration, e.g. details of archiving or snapshots
G06F 16/178 - Techniques for file synchronisation in file systems
3.
Search result replication management in a search head cluster
Systems and methods for search result replication in a search head cluster of a data aggregation and analysis system. An example method may include receiving, by a search head leader of a search head cluster including multiple search heads, from a first search head of the plurality of search heads, a search result in response to a search query. The search head leader parses a registry comprising a set of replicas of the search result in the search head cluster to determine a replication count corresponding to a number of replicas of the search result. A determination is made that the replication count is greater than a target replication count. Based on the determination, a selected replica from the set of replicas is identified based at least in part on a recency of use of the selected replica and a deletion of the selected replica is initiated.
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
A computing device can receive a query in a first query language that identifies a set of data to be processed and determine that at least a portion of the set of data resides in an external data system that uses a different query language. The query system can translate the query in the first query language in to a second query language for the external data system. To retrieve results of the translated query, the computing device may determine a quantity of results of generated by the translated query and generate one or more results readers to read the results in parallel. The computing device may further process the results and provide them to a user.
G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
5.
Automated provision of a listing of events related and corresponding attributes related to a selected event through generation of graph-based dense representations of events of a nodal graph
A computerized method is disclosed that includes operations of receiving incoming data including event data, extracting entities from the event data based on a graph ontology, generating a graph-based dense representation of each graph entity according to the graph ontology, wherein the graph-dense representations are stored in a vector database, computing relatedness scores between each of the entities, generating a listing of events related to a selected event, wherein the listing of events is ordered by corresponding relatedness scores, generating a graphical user interface illustrating the listing of events related to the selected event, and causing rendering of the graphical user interface on a display screen of a network device. Generating the graph-based dense representations may include training a graph neural network model on a corpus of metapaths to produce node embeddings.
Implementations of this disclosure provide an anomaly detection system that automatically tunes parameters of a forecasting detector that detects anomalies in a metric time series. The anomaly detection system may implement a three-stage process where a first stage tunes a historical window parameter, a second stage tunes a current window parameter, and a third stage tunes the number of standard deviation different from historical mean required to trigger an alert. The tuned historical window length determined by the first stage may be provided to the second stage as input. Both the tuned historical window length and the tuned current window length may be provided to the third stage as input as use in determining the tuned number of standard deviations.
A method of dynamic cluster manager failover includes routing data traffic associated with managing a plurality of indexers in a cluster to a first cluster manager, wherein the first cluster manager is associated with an active role and is operable to manage the plurality of indexers in the cluster. The method also includes transmitting periodic heartbeat request messages from a second cluster manager of the cluster to the first cluster manager, wherein the second cluster manager is associated with a standby role. Further, the method includes detecting, at the second cluster manager, a loss of heartbeat response messages from the first cluster manager. Also, the method includes receiving information from a set of indexers regarding a status of the first cluster manager and in response to a determination that the status of the first cluster manager is offline, promoting the second cluster manager to switch over to the active role.
Techniques and mechanisms are disclosed for configuring actions to be performed by a network security application in response to the detection of potential security incidents, and for causing a network security application to report on the performance of those actions. For example, users may use such a network security application to configure one or more “modular alerts.” As used herein, a modular alert generally represents a component of a network security application which enables users to specify security modular alert actions to be performed in response to the detection of defined triggering conditions, and which further enables tracking information related to the performance of modular alert actions and reporting on the performance of those actions.
G06F 21/55 - Detecting local intrusion or implementing counter-measures
G06F 21/53 - Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems during program execution, e.g. stack integrity, buffer overflow or preventing unwanted data erasure by executing in a restricted environment, e.g. sandbox or secure virtual machine
A computing device can receive a query that identifies a set of data to be processed and determine that a portion of the set of data resides in an external data system. The query system can request data identifiers associated with data objects of the set of data from the external data system and communicate the data identifiers to a data queue. The computing device can instruct one or more search nodes to retrieve the identifiers from the data queue. The search nodes can use the data identifiers to retrieve data objects from the external data system and process the data objects according to instructions received from the computing device. The search nodes can provide results of the processing to the computing device.
As an indexer indexes and groups events, it can generate data slices that include events. Based on a slice rollover policy, the indexer can add a particular slice to an aggregate slice. Based on an aggregate slice backup policy, the indexer can store a copy of the aggregate slice to a shared storage system. The aggregate slice can be used for restore purposes in the event the indexer fails or becomes unresponsive.
Systems and methods are described for a implementing a streaming data processing system that includes a pool of pre-configured resources and a pool of dedicated resources. The streaming data processing system can implement a processing pipeline using compute resources. The pool of pre-configured resources can support previews of processing pipelines for a plurality of users and the pool of dedicated resources can support full deployments of processing pipelines for a particular user. The streaming data processing system can implement a preview of a processing pipeline using a pre-configured resource of the pool of pre-configured resources. Further, the streaming data processing system can implement the processing pipeline using a dedicated resource of the pool of dedicated resources. The streaming data processing system can provision the dedicated resource and deploy the processing pipeline using the dedicated resource.
Systems and methods are described for a query conversion system to convert a first query string from a first version of a query language to a second version of the query language. The query conversion system may be associated with a tokenizer and parser, code converter, compatibility library, and a query formatter. The tokenizer and parser may tokenize and parse a query string to create a first node tree with commands. The code converter may parse the first node tree while using the compatibility library to convert the commands and generate a second node tree. The query formatter may create a second query string executable by the second version of the query language.
Provided are systems and methods for determining and displaying service performance information via a graphical user interface. A method can include visually rendering a service-level dashboard reflecting performance of a service and presenting a visual indication of health of each component service and a list of events each corresponding to a change in performance of one of the component services. The method can further include responsive to receiving, via a graphical user interface (GUI), a selection of a component service, visually rendering a system-level dashboard reflecting performance of the selected component-level service, wherein the component service is performed by one or more machines, and wherein the system-level dashboard presents the machines and one or more events each corresponding to a change in performance of one of the machines.
H04L 41/0631 - Management of faults, events, alarms or notifications using root cause analysisManagement of faults, events, alarms or notifications using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
H04L 43/0817 - Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters by checking availability by checking functioning
H04L 67/02 - Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
14.
Applying updated configuration dynamically to remote capture agents
The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains, at a remote capture agent, configuration information for the remote capture agent from a configuration server over a network. Next, the system uses the configuration information to configure the generation of event data from network data obtained from network packets at the remote capture agent. The system then uses the configuration information to configure transformation of the event data or the network data into transformed event data at the remote capture agent.
H04L 43/04 - Processing captured monitoring data, e.g. for logfile generation
H04L 41/046 - Network management architectures or arrangements comprising network management agents or mobile agents therefor
H04L 41/0816 - Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
H04L 41/0853 - Retrieval of network configurationTracking network configuration history by actively collecting configuration information or by backing up configuration information
H04L 43/106 - Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
15.
EXTERNALLY DISTRIBUTED BUCKETS FOR EXECUTION OF QUERIES
A data intake and query system can manage the search of data stored at an external location relative to the data intake and query system using one or more indexers. The data intake and query system can receive data stored at the external location. The data intake and query system can process the data and generate an index using the one or more indexers. The data intake and query system can discard the data and store the index and a location identifier of the external location in one or more buckets. In response to a query, the data intake and query system can identify that at least a subset of the data is responsive to the query using the index and can obtain the at least the subset of the data from the external location using the location identifier.
A query coordinator can receive and parse a query. Based on parsing the query, the query coordinator can identify one or more components of a first data processing system to obtain and process data according to the query. The query coordinator can define a query processing scheme based on identifying the one or more components and provide the query processing scheme to a second data processing system. Based on providing the query processing scheme, the query coordinator can obtain an output of the second data processing system. The query coordinator can identify a second query based on the output and provide the second query to the one or more components.
A query coordinator can receive a query. The query coordinator can determine one or more data semantics of a first data processing system. The data semantics of the first data processing system may be based on execution of one or more queries by the first data processing system. The query coordinator can define a query processing scheme for obtaining and processing data based on the query. The query processing scheme may include instructions for a second data processing system to execute at least a portion of the query according to the data semantics of the first data processing system. The query coordinator can provide the query processing scheme to the second data processing system and obtain query results from the second data processing system.
A query coordinator can receive a query and identify a first portion of the query to be processed by a first data processing system and a second portion of the query to be processed by a second data processing system. The query coordinator can obtain a modified query based on identifying the first portion and the second portion of the query. The query coordinator can define a query processing scheme according to the modified query and provide the query processing scheme to the second data processing system. Based on providing the query processing scheme, the query coordinator can obtain an output of the second data processing system. The query coordinator can identify a second query based on the output and provide the second query to a component of the first data processing system.
The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains one or more event streams from one or more remote capture agents over one or more networks, wherein the one or more event streams include event data generated from network packets captured by the one or more remote capture agents. Next, the system applies one or more transformations to the one or more event streams to obtain transformed event data from the event data. The system then enables querying of the transformed event data.
A method includes displaying events that correspond to search results of a search query, the events comprising data items of event attributes, the events displayed in a table. The table includes columns corresponding to an event attribute, rows corresponding events, cells populated data items, and interactive regions corresponding to at least one data item and selectable to add one or more commands to the search query. A reference event attribute is determined based on an analysis of a data object. A supplemental column corresponding to a supplemental event attribute is added to the table based on the reference event attribute. Supplemental interactive regions are added to the table and correspond to supplemental data items.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
Systems and methods are disclosed for providing a multi-component application, including a first and second component, and a first and second server. The first component may be implemented at the first server, while a second component may be implemented at a client device. An end user of a client device may request access to metadata stored on the second server that is utilized by the second component to implement the multi-component application. The end user may authenticate with the first component. The first component may then communicate with the second server to authenticate the end user to the second server, thereby granting the end user access to the second server without having to reauthenticate to the second server.
Resegmenting chunks of data for load balancing is disclosed. A plurality of first chunks of data is received. The plurality of first chunks of data includes one or more entries that include raw data produced by a component of an information technology environment and that reflects activity in the information technology environment. The plurality of first chunks of data is resegmented into a plurality of second chunks of data based on a source type of the plurality of first chunks. A first subset of the plurality of second chunks of data is distributed to a first indexer of a set of indexers. An occurrence of a trigger event is determined, and in response to the trigger event, a second subset of the plurality of second chunks of data is distributed to a second indexer of the set of indexers.
Embodiments described herein are directed to facilitating management and storage of configurations. In one embodiment, a request to provide configurations associated with a user-application pair is identified. Based on the user-application pair, a user-defined configuration cache is accessed to obtain user-defined configurations, an application-defined configuration cache is accessed to obtain application-defined configurations, and a system-defined configuration cache is accessed to obtain system-defined configurations. Thereafter, at least a portion of the user-defined configurations, the application-defined configurations, and the system-defined configurations are aggregated or merged to generate a set of configurations associated with the user-application pair. Such a set of configurations associated with the user-application pair are provided for use in performing a task.
In embodiments of statistics chart row mode drill down, a first interface is displayed in a table format that includes columns and rows, where each row is associated with an event and each column includes field for a respective event. The rows can further include one or more aggregated metrics representing a number of events associated with a respective row. A row can be emphasized in the first interface and, in response a menu can be displayed with selectable options to transition to a second interface, where the data displayed by the second interface is based on an option selected from the menu.
G06F 15/16 - Combinations of two or more digital computers each having at least an arithmetic unit, a program unit and a register, e.g. for a simultaneous processing of several programs
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06F 40/18 - Editing, e.g. inserting or deleting of tablesEditing, e.g. inserting or deleting using ruled lines of spreadsheets
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a patternLocating or processing of specific regions to guide the detection or recognition
G06F 9/451 - Execution arrangements for user interfaces
A process for providing requests to a management application in a multi-tenant environment is described herein. In embodiments, a broker client is deployed within a tenant execution environment executed by a server computer system. In embodiments, the broker client is configured to communicate with a broker responsible for managing the tenant execution environment based on configuration information. Furthermore, in various embodiments, request to perform operations associated with the tenant execution environment are transmitted to the broker client over a connection and the broker client provides the request to the broker for execution.
Systems and methods are disclosed involving user interface (UI) search tools for locating data, including tools for summarizing indexed raw machine data that organize and present results to enable expansion and exploration of initial summarizations. The initial summarizations may be explored and refined to help users determine how to identify and best focus a search on data subsets of greater interest.
Techniques are described for enabling users of an information technology (IT) and security operations application to create highly reusable custom functions for playbooks. The creation and execution of playbooks using an IT and security operations application generally enables users to automate operations related to an IT environment responsive to the identification of various types of incidents or other triggering conditions. Users can create playbooks to automate operations such as, for example, modifying firewall settings, quarantining devices, restarting servers, etc., to improve users' ability to efficiently respond to various types of incidents operational issues that arise from time to time in IT environments.
A data intake and query system can manage the search of large amounts of data using one or more processing nodes. The data intake and query system can identify a group of processing nodes and assign a first processing node of the group to download and search a particular data group based on a first node map. The data intake and query system may identify an action associated with the first processing node. The data intake and query system can cause a particular processing node of the group to download the particular data group based on a second node map and transmit an authorization to perform the action to the first processing node.
Systems and methods are disclosed for processing and executing queries in a data intake and query system. The data intake and query system receives raw machine data at an indexing system, and stores at least a portion of the raw machine data in buckets using containerized indexing nodes instantiated in a containerized environment. The data intake and query system stores the buckets in a shared storage system.
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
Embodiments described herein are directed to facilitating management of collection agents. In one embodiment, a control request is received at an agent service manager from an agent controller that manages collection agents that collect data. The agent controller and the collection agents operate on a remote computing machine. A desired agent event is identified to be executed in association with a set of collection agent of the collection agents. An indication of the desired agent event is provided to the agent controller for execution of the desired agent event in association with each collection agent of the set of collection agents.
One or more processing devices receive a definition of a search query for a correlation search of a data store, the data store comprising time-stamped events that each include raw machine data reflecting activity in an information technology environment and produced by a component of the information technology environment, receive a definition of a triggering condition to be evaluated based on aggregated statistics of values of one or more fields of a dataset produced by the search query, receive a definition of one or more actions to be performed when the triggering condition is satisfied, generate, using search processing language, a statement to define the search query and the triggering condition, and in view of the results of the execution of the search processing language, cause generation of the correlation search using the defined search query, the triggering condition, and the one or more actions, the correlation search comprising updated search processing language having the search query and a processing command for criteria on which the triggering condition is based.
A visualization can include a set of swim lanes, each swim lane representing information about an event type. An event type can be specified, e.g., as those events having certain keywords and/or having specified value(s) for specified field(s). The swim lane can plot when (within a time range) events of the associated event type occurred. Specifically, each such event can be assigned to a bucket having a bucket time matching the event time. A swim lane can extend along a timeline axis in the visualization, and the buckets can be positioned at a point along the axis that represents the bucket time. Thus, the visualization may indicate whether events were clustered at a point in time. Because the visualization can include a plurality of swim lanes, the visualization can further indicate how timing of events of a first type compare to timing of events of a second type.
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
Embodiments are disclosed for a data analysis tool for facilitating iterative and exploratory analysis of large sets of data. In some embodiments a data analysis tool includes a graphical user interface through which an interactive set of field identifiers is displayed. Each of the listed field identifiers may reference fields associated with a set of events returned in response to a search query, the set of events including machine data produced by components within an information technology (IT) environment that reflects activity in the IT environment. In response to user selections of field identifiers included in the displayed set, a data analysis tool may cause display of manipulable visualizations based on values included in fields referenced by the selected field identifiers.
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04845 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
Provided are systems and methods for indicating deployment of application features. In one embodiment, a method is provided that includes determining available features of a current deployment of an application for receiving machine-generated data from one or more data sources of a data system, determining un-deployed features of the current deployment of the application, wherein the un-deployed features comprise one or more of the available features that is configured to use input data from a data source and wherein the input data is not available to the feature in the current deployment of the application, and causing display of a deployment graphical user interface (GUI) that comprises an indication of the un-deployed features.
A disclosed computer-implemented method includes receiving and indexing the raw data. Indexing includes dividing the raw data into time stamped searchable events that include information relating to computer or network security. Store the indexed data in an indexed data store and extract values from a field in the indexed data using a schema. Search the extracted field values for the security information. Determine a group of security events using the security information. Each security event includes a field value specified by a criteria. Present a graphical interface (GI) including a summary of the group of security events, other summaries of security events, and a remove element (associated with the summary). Receive input corresponding to an interaction of the remove element. Interacting with the remove element causes the summary to be removed from the GI. Update the GI to remove the summary from the Gl.
Systems and methods are described for distributed processing a query in a first query language utilizing a query execution engine intended for single-device execution. While distributed processing provides numerous benefits over single-device processing, distributed query execution engines can be significantly more difficult to develop that single-device engines. Embodiments of this disclosure enable the use of a single-device engine to support distributed processing, by dividing a query into multiple stages, each of which can be executed by multiple, concurrent executions of a single-device engine. Between stages, data can be shuffled between executions of the engine, such that individual executions of the engine are provided with a complete set of records needed to implement an individual stage. Because single-device engines can be significantly less difficult to develop, use of the techniques described herein can enable a distributed system to rapidly support multiple query languages.
G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
An identify resolution system performs actions comprises a set-up process and an identity resolution process that executes asynchronously with respect to the set-up process. the set-up process includes accessing machine data including a plurality of event data objects, each event data object of the plurality of event data objects including timestamped raw machine-generated data indicative of performance or operation of one or more entities in a computer network environment. The identity resolution process ascertains the identity of an entity associated with the computer network environment, based on the association data in the data store, wherein the identity of the entity is not expressed directly in the association data in the data store.
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
H04L 67/146 - Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding
38.
Supplementing extraction rules based on event clustering
Systems and methods include causing presentation of a first cluster in association with an event of the first cluster, the first cluster from a first set of clusters of events. Each event includes a time stamp and event data. Based on the presentation of the first cluster, an extraction rule corresponding to the event of the first cluster is received from a user. Similarities in the event data between the events are determined based on the received extraction rule. The events are grouped into a second set of clusters based on the determined similarities. Presentation is caused of a second cluster in association with an event of the second cluster, where the second cluster is from the second set of clusters.
A process for facilitating downscaling of datastores (e.g., in a stateful system) is described herein. In embodiments, a set of metrics associated with a set of data stores of a stateful service is obtained. The set of metrics may indicate a utilization of each of the data stores of the set of data stores. Based on the set of metrics indicating underutilization associated with at least a portion of the set of data stores, a determination is made to initiate a downscaling of the set of data stores. Thereafter, a downscaler is deployed to perform downscaling operations to downscale the set of data stores. The downscaler communicates with a first data store to replicate data of the first data store onto a second data store. Based on identifying that the downscaler has completed the downscaling operations to downscale the set of data stores, the downscaler is terminated.
Embodiments of the present invention are directed to facilitating efficient message queueing. In particular, embodiments herein describe, among other things, a redelivery monitor used to monitor when to redeliver messages, or tasks, for reprocessing based on expiration of a redelivery deadline. In this regard, markers indicating processing states for tasks being processed are read by the redelivery monitor. When the processing state indicates that processing is ongoing, the redelivery deadline is extended such that a message or task is not redelivered for processing while the message or task is being processed.
Embodiments described herein are directed to facilitating performing online data decomposition to identify multiple seasonal components. In accordance with aspects of the present disclosure, a first iterative process is performed to determine a first seasonal component associated with an incoming data point based on a set of previous data points of a time series data set and corresponding data components. In addition, a second iterative process is performed to determine a second seasonal component associated with the incoming data point based on previous data points of the time series data set and corresponding data components. The first seasonal component and the second seasonal component can then be provided for analysis of the incoming data point (e.g., for presentation, for use in determining trend and residual components, etc.).
Embodiments of the present disclosure are directed to facilitating performing online data forecasting. In operation, data decomposition of an incoming data point is performed to determine a trend component associated with the incoming data point. Such a trend component, and previous trend components, can be used to determine a trend component expected for a data point subsequent to the incoming data point. A seasonality component expected for the data point subsequent to the incoming data point can be identified, for example, based on a seasonality component associated with a previous corresponding data point. Thereafter, the expected trend and seasonality components can be used to predict the data point subsequent to the incoming data point. Such a data prediction can be performed in an online processing manner such that a subsequent data point is not used to decompose the incoming data point or forecast the data point.
G06F 18/214 - Generating training patternsBootstrap methods, e.g. bagging or boosting
G06Q 10/04 - Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
The disclosed embodiments include a method performed by a data intake and query system. The method includes receiving a search query by a search head, defining a search process for applying the search query to indexers, delegating a first portion of the search process to indexers and a second portion of the search process to intermediary node(s) communicatively coupled to the search head and the indexers. The first portion can define a search scope for obtaining partial search results of the indexers and the second portion can define operations for combining the partial search results by the intermediary node(s) to produce a combination of the partial search results. The search head then receives the combination of the partial search results, and outputs final search results for the search query, where the final search results are based on the combination of the partial search results.
Multiple storage system event handling includes obtaining multiple events for storage on multiple storage systems. For each of the multiple events, field values from each event are extracted. The field values are matched to configurations of the storage systems to identify a subset of the storage system having a matching configuration. The event is transmitted to the subset. The multiple events are transmitted to heterogeneous subsets.
A schema consistency mechanism monitors data ingested by a data intake and query system for changes to the structure, or data schema, associated with the data. A schema consistency monitor obtains data from a data source (or, more generally, from any number of separate data sources) at a plurality of points in time. The data is analyzed to determine whether a first portion of the data received at a first point in time conforms to a first data schema and that a second portion of the data received at a second point in time conforms to a second data schema that is different from the first data schema (thereby indicating a change to the associated data schema). A graphical user interface (GUI) can be generated that includes indications of identified changes to one or more data schemas associated with data.
A method includes causing display to a user of at least one event of a first result set from a first pipelined search on events at an event source. Each event comprises a time stamp and a portion of machine data. A selection of a command is received from the user. The selection is to extend the first pipelined search with the selected command in a second pipelined search. The system selects between the first result set and the event source for execution of the second pipelined search based on an analysis of the selected command and at least one command of the first pipelined search. Based on the selecting being of the first result set, display to the user is caused of at least one event of a second result set from the execution of the second pipelined search on the first result set.
A quality score for a computer application release is determined using a first number of unique users who have launched the computer application release on user devices and a second number of unique users who have encountered at least once an abnormal termination with the computer application release on user devices. Additionally or optionally, an application quality score can be computed for a computer application based on quality scores of computer application releases that represent different versions of the computer application. Additionally or optionally, a weighted application quality score can be computed for a computer application by further taking into consideration the average application quality score and popularity of a plurality of computer applications.
Ingest health monitoring includes receiving an event stream of events in a data intake and query system to store on at least one storage system and obtaining an event from the event stream. Ingest health monitoring further includes transmitting the event to a selected ingest module queue for the event, updating an output rate indicator counter for the selected ingest module queue when failure to store the event in the ingest module queue occurs, obtaining the event from the selected ingest module queue, processing the event to generate a file for the event, and transmitting the file to the at least one storage system. Ingest health monitoring further includes updating the write failure indicator counter for a storage system of the at least one storage system when failure to transmit to the storage system occurs and updating the user interface based on the output rate indicator counter and the write failure indicator counter.
Embodiments are directed towards real time display of event records and extracted values based on at least one extraction rule, such as a regular expression. A user interface may be employed to enable a user to have an extraction rule automatically generate and/or to manually enter an extraction rule. The user may be enabled to manually edit a previously provided extraction rule, which may result in real time display of updated extracted values. The extraction rule may be utilized to extract values from each of a plurality of records, including event records of unstructured machine data. Statistics may be determined for each unique extracted value, and may be displayed to the user in real time. The user interface may also enable the user to select at least one unique extracted value to display those event records that include an extracted value that matches the selected value.
G06F 7/00 - Methods or arrangements for processing data by operating upon the order or content of the data handled
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
Systems and methods are disclosed for receiving, at query coordinator, a search query. The query coordinator parses the search query and generates tasks for different runtime systems. The query coordinator configures an interface enabling inter system communication between the runtime systems. The generated tasks are distributed to the runtime systems and partial results of a runtime system are communicated to the interface. The query coordinator retrieves the partial results from the interface, finalizes the partial results, and sends finalized results them to the requestor of the search query.
In the disclosed embodiments, a remote capture agent monitors network packets traversing a network interface of a computing device in an information technology environment. Network data is obtained from the network packets. The network data is modified based on configuration information obtained by the remote capture agent from a configuration server to obtain modified network data. Timestamped events are generated based on the modified network data, and the timestamped events are sent to another component on the network for subsequent processing.
Systems and methods are disclosed for executing a query that includes an indication to process data managed by an external data system. The system identifies the external data system that manages the data to be processed and generates a subquery for the external data system indicating that the results of the subquery are to be sent to one worker node of multiple worker nodes. The system instructs the one worker node to distribute the results received from the external data system to multiple worker nodes for processing.
The disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system causes for display a graphical user interface (GUI) for configuring the generation of time-series event data from network packets captured by one or more remote capture agents. Next, the system causes for display, in the GUI, a first set of user-interface elements containing a set of statistics associated with one or more event streams that comprise the time-series event data. The system then causes for display, in the GUI, one or more graphs comprising one or more values from the set of statistics. Finally, the system causes for display, in the GUI, a value of a statistic from the set of statistics based on a position of a cursor over the one or more graphs.
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
G06F 3/0481 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/26 - Visual data miningBrowsing structured data
H04L 41/0813 - Configuration setting characterised by the conditions triggering a change of settings
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
H04L 67/12 - Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
H04L 67/75 - Indicating network or usage conditions on the user display
54.
Generating new visualizations based on prior journey definitions
Systems, methods, and computer readable media are disclosed for generating and providing concurrent journey visualizations associated with different journey definitions. In computer-implemented embodiments, a data intake and query system, or a journey visualization computing tool, can be used to generate and provide concurrent representations corresponding with different journey definitions. In operation, a set of journey instances associated with a journey having a set of steps is obtained. Each step may be associated with at least one event that includes raw machine data produced by a component of an information technology environment. Upon obtaining different journey definitions specifying filters to apply to the set of journey instances, the data intake and query system can generate journey visualizations in accordance with the journey definitions. Thereafter, the journey visualizations corresponding with the journey definitions can be concurrently displayed by a computing device via a graphical user interface.
A process for facilitating autoscaling in a stateful system is described herein. In embodiments, a set of metrics associated with a set of components of a stateful service is obtained. The set of metrics may generally indicate a utilization or a load of each of the components of the set of components (e.g., message managers and/or data stores). Thereafter, it is determined to initiate a scaling event at the stateful service in association with the set of components of the stateful service based on at least a portion of the set of metrics attaining a metric threshold indicating a threshold value for determining whether to scale stateful service components. A scaling request can then be provided to the stateful service to initiate the scaling event at the stateful service in association with the set of components of the stateful service.
G06F 9/50 - Allocation of resources, e.g. of the central processing unit [CPU]
G06F 11/14 - Error detection or correction of the data by redundancy in operation, e.g. by using different operation sequences leading to the same result
G06F 11/20 - Error detection or correction of the data by redundancy in hardware using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
G06F 11/34 - Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation
56.
Query execution using access permissions of queries
A method includes assigning an access permission of a first user to a query object that represents a first query, the access permission granting the first user access rights to one or more data sources of the first query the access permission being assigned as a runtime permission of the first query, granting a request from a second user to execute a second query, the first query being a subquery of the second query, and allowing the second user to execute the first query on the one or more data sources of the first query using the runtime permission assigned to the first query in executing the second query using the first query as the subquery.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
In a computer-implemented method for configuring a distributed computer system comprising a plurality of nodes of a plurality of node classes, configuration files for a plurality of nodes of each of the plurality of node classes are stored in a central repository. The configuration files include information representing a desired system state of the distributed computer system, and the distributed computer system operates to keep an actual system state of the distributed computer system consistent with the desired system state. The plurality of node classes includes forwarder nodes for receiving data from an input source, indexer nodes for indexing the data, and search head nodes for searching the data. Responsive to receiving changes to the configuration files, the changes are propagated to nodes of the plurality of nodes impacted by the changes based on a node class of the nodes impacted by the changes.
H04L 41/0813 - Configuration setting characterised by the conditions triggering a change of settings
H04L 41/084 - Configuration by using pre-existing information, e.g. using templates or copying from other elements
H04L 41/0853 - Retrieval of network configurationTracking network configuration history by actively collecting configuration information or by backing up configuration information
H04L 67/00 - Network arrangements or protocols for supporting network services or applications
H04L 67/06 - Protocols specially adapted for file transfer, e.g. file transfer protocol [FTP]
H04L 69/329 - Intralayer communication protocols among peer entities or protocol data unit [PDU] definitions in the application layer [OSI layer 7]
58.
Processing distributed jobs using multiple connections
A method to assist with processing distributed jobs by retrieving and/or synchronizing supplemental job data. The method includes: transmitting, by a first component of a first execution environment via a first connection to a second component of a second execution environment, a first request associated with a job; receiving, by the first component via a second connection from the second component, a second request associated with the job; transmitting, by the first component via the second connection to the second component, a response to the second request, the response comprising an information associated with the job; and receiving, by the first component via the first connection from the second component, a result of the job.
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
H04L 41/0896 - Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities
H04L 65/65 - Network streaming protocols, e.g. real-time transport protocol [RTP] or real-time control protocol [RTCP]
H04L 67/02 - Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
59.
Assigning field values based on an identified extraction rule
In embodiments, method and systems are provided for facilitating identification of field values based on an extraction rule. In some implementations, a graphical user interface receives first input identifying an extraction mode that defines at least a portion of how to extract values from fields of events, and further receives second input configuring an extraction rule that defines at least a first field of the fields based on the extraction mode. The second input may include selecting a delimiter type for a delimiter mode, or specifying fields from a sample event for a regular expression mode. As such, an extraction rule may be configured, and a first set of the values from the events may be assigned to the first field based on the extraction rule.
Provided are systems and methods for managing storage of machine data. In one embodiment, a method can be provided. The method can include receiving, from one or more data sources, raw machine data; processing the raw machine data to generate processed machine data; storing the processed machine data in a data store; and determining an allocated data size associated with the processed machine data stored in the data store, wherein the allocated data size is the size of the raw machine data corresponding to the processed machine data stored in the data store.
G06Q 20/10 - Payment architectures specially adapted for electronic funds transfer [EFT] systemsPayment architectures specially adapted for home banking systems
In some embodiments, a method may include display of a data summary view of a set of events that correspond to query results of a query. Each event of the set of events may include data items of a plurality of event attributes. In embodiments, the data summary view can include various summary reports. Each summary report can include summary entries and a summary graph that each present a summary of data items of a selected event attribute, of the plurality of event attributes. At least one summary report can include summary entries that are selectable by a user. The method may further include filtering the set of event, in response to, and based on, selection of one or more of the selectable summary entries by the user and updating of at least the first and second summary graphs to correspond to the filtered set of events.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
A method includes receiving, in a first query interface, a query composed by the user by typing commands into a query box of the first query interface and based on the receiving of the query, causing events corresponding to query results of the query to be displayed in the first query interface with fields corresponding to the events. Based on the selection by the user of an option, a second query interface is displayed with a table that includes events that correspond to query results of a loaded query. The table includes columns corresponding to event attributes, rows corresponding to events. Cells are populated with the data items of event attributes, where one of the columns corresponds to a field of the fields displayed in the first query interface. The table also includes interactive regions selectable by the user to add one or more commands to the loaded query.
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04842 - Selection of displayed objects or displayed text elements
G06F 16/00 - Information retrievalDatabase structures thereforFile system structures therefor
Embodiments of the present invention are directed to facilitating performing data extraction via efficient extraction rule matching. Generally, an extraction rule can be determined to match an event based on a two-step process. In particular, initially, a determination that a set of fixed substrings associated with the extraction rule matches fixed substrings of the event can be made. Based on fixed substring match, a determination can be made that a set of fields associated with the extraction rule matches fields of the event. In such a case, the extraction rule can be deemed to match the event and used to extract values from the event.
A continuous anomaly detection service receives data stream and performs continuous anomaly detection on the incoming data streams. This continuous anomaly detection is performed based on anomaly detection definitions, which define a signal used for anomaly detection and an anomaly detection configuration. These anomaly detection definitions can be modified, such that continuous anomaly detection continues to be performed for the data stream and the signal, based on the new anomaly detection definition.
The technology disclosed relates to formulating and refining field extraction rules that are used at query time on raw data with a late-binding schema. The field extraction rules identify portions of the raw data, as well as their data types and hierarchical relationships. These extraction rules are executed against very large data sets not organized into relational structures that have not been processed by standard extraction or transformation methods. By using sample events, a focus on primary and secondary example events help formulate either a single extraction rule spanning multiple data formats, or multiple rules directed to distinct formats. Selection tools mark up the example events to indicate positive examples for the extraction rules, and to identify negative examples to avoid mistaken value selection. The extraction rules can be saved for query-time use, and can be incorporated into a data model for sets and subsets of event data.
An event limited field picker for a search user interface is described. In one or more implementations, a service may operate to collect and store data as events each of which includes a portion of the data correlated with a point in time. Clients may use a search user interface perform searches by input of search criteria. Responsive to receiving search criteria, the service may operate to apply a late binding schema to extract events that match the search criteria and provide search results for display via the search user interface. The search user interface exposes an event limited field picker operable to make selections of fields with respect to individual events in a view of the search results. In response to receiving an indication of a fields selected via the picker, visibility of selected fields may be updated to control which field and values are included in different views.
G06F 16/25 - Integrating or interfacing systems involving database management systems
G06F 3/04817 - Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/04842 - Selection of displayed objects or displayed text elements
Embodiments of the present invention are directed to facilitating data preprocessing for machine learning. In accordance with aspects of the present disclosure, a training set of data is accessed. A preprocessing query specifying a set of preprocessing parameter values that indicate a manner in which to preprocess the training set of data is received. Based on the preprocessing query, a preprocessing operation is performed to preprocess the training set of data in accordance with the set of preprocessing parameter values to obtain a set of preprocessed data. The set of preprocessed data can be provided for presentation as a preview. Based on an acceptance of the set of preprocessed data, the set of preprocessed data is used to train a machine learning model that can be subsequently used to predict data.
Techniques are described for providing a highly available data ingestion system for ingesting machine data sent from remote data sources across potentially unreliable networks. To provide for highly available delivery of such data, a data intake and query system provides users with redundant sets of ingestion endpoints to which messages sent from users' computing environments can be delivered to the data intake and query system. Users' data sources, or data forwarding components configured to obtain and send data from one or more data sources, are then configured to encapsulate obtained machine data into discrete messages and to send copies of each message to two or more of the ingestion endpoints provisioned for a user. The ingestion endpoints receiving the messages implement a deduplication technique and provide only one copy of each message to a subsequent processing component (e.g., to an indexing subsystem for event generation, event indexing, etc.).
This technology is directed to facilitating scalable and secure data collection. In particular, scalability of data collection is enabled in a secure manner by, among other things, abstracting a connector(s) to a pod(s) and/or container(s) that executes separate from other data-collecting functionality. For example, an execution manager can initiate deployment of a collect coordinator on a first pod associated with a first job and deployment of a first connector on a second pod associated with a second job separate from the first job of a container-managed platform. The collect coordinator can provide a data collection task to the first connector deployed on the second pod of the second job. The first connector can then obtain the set of data from the data source and provide the set of data to the collect coordinator for providing the set of data to a remote source.
The disclosed embodiments provide a system that processes network data. During operation, the system obtains, at a remote capture agent, a first protocol classification for a first packet flow captured by the remote capture agent. Next, the system uses configuration information associated with the first protocol classification to build a first event stream from the first packet flow at the remote capture agent, wherein the first event stream comprises time-series event data generated from network packets in the first packet flow based on the first protocol classification. The system then transmits the first event stream over a network for subsequent storage and processing of the first event stream by one or more components on the network.
Disclosed is a data fabric service system that can be implemented in a distributed computer network, such as a data intake and query system. The data index and query system can receive a search query and define a search scheme for applying the search query on distributed data storage systems including internal data storage and external data storage. The data index and query system may provide a portion of the search scheme to a search service of the data fabric service system, which can cause worker nodes of the data fabric service system to perform various functions—including applying the search query to the external data storage based on the portion of the search scheme in order to obtain search results.
G06F 16/25 - Integrating or interfacing systems involving database management systems
G06F 16/27 - Replication, distribution or synchronisation of data between databases or within a distributed database systemDistributed database system architectures therefor
An actionable event collector in a server cluster receives information specifying an actionable event instance regarding an actionable event occurrence in the server cluster. The actionable event collector transmits a representation of the actionable event instance to an actionable event queue builder. The actionable event queue builder inserts the representation as an entry into an actionable event queue. The event action dispatcher processes the entry from the actionable event queue, wherein processing the entry comprises determining a responsive action for the entry and causing performance of the responsive action.
H04L 41/0631 - Management of faults, events, alarms or notifications using root cause analysisManagement of faults, events, alarms or notifications using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
G06F 16/2458 - Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
G06F 9/451 - Execution arrangements for user interfaces
Operational machine components of an information technology (IT) or other microprocessor- or microcontroller-permeated environment generate disparate forms of machine data. Network connections are established between these components and processors of an automatic data intake and query system (DIQS). The DIQS conducts network transactions on a periodic and/or continuous basis with the machine components to receive the disparate data and ingest certain of the data as measurement entries of a DIQS metrics datastore that is searchable for DIQS query processing. The DIQS may receive search queries to process against the received and ingested data via an exposed network interface. In one example embodiment, a query building component conducts a user interface using a network attached client device. The query building component may elicit search criteria via the user interface using a natural language interface, construct a proper query therefrom, and present new information based on results returned from the DIQS.
An information technology (IT) and security operations application enables the automatic assignment of incident events to analysts based on a variety of characteristics of the incident events to be assigned, the analysts and analyst teams, and other considerations. An IT and security operations application can perform the automatic assignment of incident events based at least in part on data indicating each analyst's knowledge of certain types of incidents, data indicating each analyst's efficiency at responding to certain types of incidents, and the like, where such data is automatically created and maintained by the application. In this manner, incident events can be efficiently assigned to analysts upon their receipt by the system without the need for a security team to constantly perform a cumbersome incident event assignment process based on a limited set of data, thereby improving analyst teams' ability to efficiently ensure the operation and security of IT environments for which the teams are responsible.
Embodiments are directed towards managing and tracking item identification of a plurality of items to determine if an item is a new or existing item, where an existing item has been previously processed. In some embodiments, two or more item identifiers may be generated. In one embodiment, generating the two or more item identifiers may include analyzing the item using a small item size characteristic, a compressed item, or for an identifier collision. The two or more item identifiers may be employed to determine if the item is a new or existing item. In one embodiment, the two or more item identifiers may be compared to a record about an existing item to determine if the item is a new or existing item. If the item is an existing item, then the item may be further processed to determine if the existing item has actually changed.
Embodiments of the present invention are directed to facilitating event forecasting. In accordance with aspects of the present disclosure, a set of events determined from raw machine data is obtained. The events are analyzed to identify leading indicators that indicate a future occurrence of a target event, wherein the leading indicators occur during a search period of time the precedes a warning period of time, thereby providing time for an action to be performed prior to an occurrence of a predicted target event. At least one of the leading indicators is used to predict a target event. An event notification is provided indicating the prediction of the target event.
Disclosed herein is a fraud analysis data reduction technique. When reviewing a large set of data for potential fraudulent action there is often too much data for a human to reasonably analyze. A technique to reduce the overall amount of data associates entities that have duplicate values stored in corresponding data elements with one another and removes those entities that do not have at least one duplicate value. The entities with duplicate values are entered into a node graph and analyzed for connected components. The connected components analysis and a duplicate threshold analysis provide usable results to identify fraudulent activity.
Embodiments described herein are directed to facilitating management of collection agents. In one embodiment, a control request is provided to an agent service manager from an agent controller that manages collection agents that collect data. The agent controller and the collection agents operate on a computing machine remote from the agent service manager. In response to the control request, a control directive is received, the control directive including an agent event indicator indicating an agent event to be executed in association with a set of collection agents of the collection agents. Thereafter, execution of the agent event is initiated in association with each collection agent of the set of collection agents.
A method includes selecting, from content packs in a centralized content management system, a content pack to update in a data intake and query system. The content pack includes utility objects. For each utility object of at least a subset of the utility objects determining whether the utility object already exists in the data intake and query system, and loading the utility object to the data intake and query system when the utility object does not exist to obtain an updated utility object. The method further includes monitoring, by the data intake and query system, an endpoint of an endpoint type using the updated utility object.
An information technology (IT) and security operations application is described that enables cross-tenant analyses of data to derive insights that can be used to provide actionable information across the application including, for example, action recommendations, threat confidence scores, and other incident data enrichments. The generation and presentation of such information to users of an IT and security operations application can enable analyst teams to more efficiently and accurately respond to various types of incidents in IT environments, thereby improving the overall operation and security of the IT environments. Furthermore, because of the shared use of an IT and security operations application concurrently by any number of separate tenants, such cross-tenant analyses can be performed in near real-time and on an ongoing basis to deliver relevant insights.
Dashboard evaluation includes receiving a dashboard code defining a dashboard that includes visualizations in a layout, rendering, in a graphical user interface (GUI) of a dashboard editing tool, the dashboard based on the dashboard code, and extracting, using the dashboard code, a data attribute of a data object represented by a visualization of the multiple visualizations. Dashboard evaluation further includes evaluating, by the dashboard editing tool, the visualization based on the data attribute to obtain a score, presenting, in the GUI of the dashboard editing tool, a recommendation based on the score failing to satisfy a first threshold, receiving, through the GUI of the dashboard editing tool and after presenting the recommendation, an edit to the dashboard code that adjusts the visualization, and storing, by the dashboard editing tool, the edit to the dashboard code.
An information technology (IT) and security operations application is described that stores data reflecting customizations that users make to GUIs displaying information about various types of incidents, and further uses such data to generate “popular” interface profiles indicating popular GUI modifications. The analysis of the GUI customizations data is performed using data associated with multiple tenants of the IT and security operations application to develop profiles that may represent a general consensus on a collection and arrangement of interface elements that enable analysts to efficiently respond to certain types of incidents. Users of the IT and security operations application can then optionally apply these popular interface profiles to various GUIs during their use of the application. Among other benefits, the ability to generate and provide popular interface profiles can help analysts and other users more efficiently investigate and respond to a wide variety of incidents within IT environments, thereby improving the operation and security of those environments.
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 16/335 - Filtering based on additional data, e.g. user or group profiles
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
In embodiments of field value search drill down, a search system exposes a search interface that displays one or more events returned as a search result set. A field-value pair can be emphasized in the field-value pairs of an event displayed in the search interface, and a menu is displayed with search options that are selectable to operate on the emphasized field-value pair of the event. The menu includes the search options to add search criteria of the emphasized field-value pair to a search command in a search bar of the search interface, exclude the search criteria of the emphasized field-value pair from a search, or create a new data search based on the emphasized field-value pair. A selection of one of the search options in the menu can be received, and the search command in the search bar is updated based on the search option that is selected.
G06F 40/18 - Editing, e.g. inserting or deleting of tablesEditing, e.g. inserting or deleting using ruled lines of spreadsheets
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a patternLocating or processing of specific regions to guide the detection or recognition
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
G06F 9/451 - Execution arrangements for user interfaces
84.
Graphical user interface for extracting from extracted fields
First one or more values are extracted from a plurality of events using a first extraction rule. The extracted first one or more values are assigned to a first field of the plurality of events as a first set of field-data item pairs. Second one or more values are extracted from the plurality of the events using a second extraction rule. The second extraction rule identifies the second one or more values and a field label corresponding to the second one or more values in the extracted first one or more values of the first set of field-data item pairs. The extracted second one or more values are assigned to a second field of the plurality of events as a second set of field-data item pairs. The field label extracted using the second extraction rule or a modified version thereof may be assigned to the second field.
Various methods and systems for tracking incomplete purchases in correlation with application performance, such as application errors or crashes, are provided. In this regard, aspects of the invention facilitate monitoring transaction and application error events and analyzing data associated therewith to identify data indicating an impact of incomplete purchases in relation to an error(s) such that application performance can be improved. In various implementations, application data associated with an application installed on a mobile device is received. The application data is used to determine that an error that occurred in association with the application installed on the mobile device correlates with an incomplete monetary transaction initiated via the application. Based on the error correlating with the incomplete monetary transaction, a transaction attribute associated with the error is determined.
A method, system, and processor-readable storage medium are directed towards generating a report derived from data, such as event data, stored on a plurality of distributed nodes. In one embodiment the analysis is generated using a “divide and conquer” algorithm, such that each distributed node analyzes locally stored event data while an aggregating node combines these analysis results to generate the report. In one embodiment, each distributed node also transmits a list of event data references associated with the analysis result to the aggregating node. The aggregating node may then generate a global ordered list of data references based on the list of event data references received from each distributed node. Subsequently, in response to a user selection of a range of global event data, the report may dynamically retrieve event data from one or more distributed nodes for display according to the global order.
H04L 41/0604 - Management of faults, events, alarms or notifications using filtering, e.g. reduction of information by using priority, element types, position or time
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
H04L 67/1097 - Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
87.
Generating event streams including modified network data monitored by remote capture agents
The disclosed embodiments provide a method and system for processing network data. During operation, the system obtains, at a remote capture agent, configuration information for the remote capture agent from a configuration server over a network. Next, the system uses the configuration information to configure the generation of event data from network data obtained from network packets at the remote capture agent. The system then uses the configuration information to configure transformation of the event data or the network data into transformed event data at the remote capture agent.
H04L 43/04 - Processing captured monitoring data, e.g. for logfile generation
H04L 41/0853 - Retrieval of network configurationTracking network configuration history by actively collecting configuration information or by backing up configuration information
H04L 41/046 - Network management architectures or arrangements comprising network management agents or mobile agents therefor
H04L 41/0816 - Configuration setting characterised by the conditions triggering a change of settings the condition being an adaptation, e.g. in response to network events
H04L 43/106 - Active monitoring, e.g. heartbeat, ping or trace-route using time related information in packets, e.g. by adding timestamps
88.
Animated visualizations of network activity across network address spaces
Techniques and mechanisms are disclosed for generating visualizations which graphically depict network activity occurring between pairs of networked computing devices. The visualizations are based on data indicating the network activity, where the network activity can involve devices having any network addresses within an entire network address space (e.g., any address within the Internet Protocol version v4 (IPv4) or IPv6 network address space), or within some subset of an entire network address space. The ability to visualize high-level information related to network activity occurring across an entire network address space enables network analysts and other users to readily analyze characteristics of computer networks which otherwise might not be evident or difficult to obtain using other types of visualizations.
As described herein, a portion of machine data of a message may be analyzed to infer, using an inference model, a sourcetype of the message. The portion of machine data may be generated by one or more components in an information technology environment. Based on the inference, a set of extraction rules associated with the sourcetype may be selected. Each extraction rule may define criteria for identifying a sub-portion of text from the portion of machine data of the message to produce a value. The set of extraction rules may be applied to the portion of machine data of the message to produce a result set that indicates a number of values identified using the set of extraction rules. Based on the result set, at least one action may be performed on one or more of inference data associated with the inference model and one or more messages.
According to embodiments, a method for navigating clusters of a data structure includes gathering data from the data structure by instrumenting instances of application software executing on the data structure. The method also includes identifying clusters of the data structure based on the gathered data. The method also includes causing display of a cluster map of the data structure, the cluster map comprising a plurality of clusters, each cluster of the plurality of clusters comprising a plurality of nodes, each node of the plurality of nodes comprising a plurality of pods, each pod of the plurality of pods comprising a plurality of containers. The method also includes providing a status for each node, each pod, and each container of each cluster. The method also includes causing display of analysis of each cluster of the cluster map, the analysis comprising granular information for each cluster.
G06F 16/28 - Databases characterised by their database models, e.g. relational or object models
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
91.
Extraction rule determination based on user-selected text
Based on a selection by a user of first one or more values of one or more events displayed in a graphical interface, an extraction rule is automatically determined that is capable of extracting a field label-value pair at least partially within at least the selected one or more values. An option is displayed that correspond to the determined extraction rule in the graphical interface. Based on the user selecting the option in the graphical interface, display is caused of second one or more values of one or more field label-value pairs extracted from the one or more events using the extraction rule. The one or more events may be displayed in a table format, and the first one or more value may be selected by the user selecting one or more cells, columns, or text portions in the table format.
Provided are systems and methods for verifying user credentials for performing a search. Verifying user credentials include receiving a search request at a search server, determining, at the search server, whether a set of user credentials of a user has been updated within a threshold period of time. The set of user credentials are received from an identity provider server and cached at the search server. Responsive to determining that the cached set of user credentials have not been updated within the threshold period of time, the identity provider server is queried for a current set of user credentials associated with the user. The current set of user credentials from the identity provider server, and used to determine that the user is authorized to perform the search. The search of the datastore is launched responsive to determining that the user is authorized.
Embodiments are directed towards real time display of event records and extracted values based on at least one extraction rule, such as a regular expression. A user interface may be employed to enable a user to have an extraction rule automatically generate and/or to manually enter an extraction rule. The user may be enabled to manually edit a previously provided extraction rule, which may result in real time display of updated extracted values. The extraction rule may be utilized to extract values from each of a plurality of records, including event records of unstructured machine data. Statistics may be determined for each unique extracted value, and may be displayed to the user in real time. The user interface may also enable the user to select at least one unique extracted value to display those event records that include an extracted value that matches the selected value.
G06Q 10/0637 - Strategic management or analysis, e.g. setting a goal or target of an organisationPlanning actions based on goalsAnalysis or evaluation of effectiveness of goals
A graphical user interface allows a customer to specify delimiters and/or patterns that occur in event data and indicate the presence of a particular field. The graphical user interface applies a customer's delimiter specifications directly to event data and displays the resulting event data in real time. Delimiter specifications may be saved as configuration settings and systems in a distributed setting may use the delimiter specifications to extract field values as the systems process raw data into event data. Extracted field values are used to accelerate search queries that a system receives.
A security platform employs a variety techniques and mechanisms to detect security related anomalies and threats in a computer network environment. The security platform is “big data” driven and employs machine learning to perform security analytics. The security platform performs user/entity behavioral analytics (UEBA) to detect the security related anomalies and threats, regardless of whether such anomalies/threats were previously known. The security platform can include both real-time and batch paths/modes for detecting anomalies and threats. By visually presenting analytical results scored with risk ratings and supporting evidence, the security platform enables network security administrators to respond to a detected anomaly or threat, and to take action promptly.
G06V 10/22 - Image preprocessing by selection of a specific region containing or referencing a patternLocating or processing of specific regions to guide the detection or recognition
G06F 3/0482 - Interaction with lists of selectable items, e.g. menus
G06F 3/0484 - Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
G06F 3/04847 - Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
H04L 41/0893 - Assignment of logical groups to network elements
H04L 43/062 - Generation of reports related to network traffic
H04L 43/045 - Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
H04L 43/08 - Monitoring or testing based on specific metrics, e.g. QoS, energy consumption or environmental parameters
G06F 3/04842 - Selection of displayed objects or displayed text elements
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
Embodiments of the present invention are directed to enhancing extraction rules utilizing user feedback. In embodiments, a set of extraction rules relevant to an event set are provided for display. Thereafter, a selection of an extraction rule is received and, in response, a set of events matching the selected extraction rule is provided for display. A modification, for example provided by a user, in association with the extraction rule or the set of events is received. Such a modification is then used (e.g., via machine learning) to enhance extraction rules available for performing subsequent data extraction.
The disclosed embodiments provide a system that facilitates the processing of network data. During operation, the system causes for display, on a computer system, a graphical user interface (GUI) for obtaining configuration information for configuring the generation of time-series event data from network packets captured by one or more remote capture agents. Next, the system causes for display, in the GUI, a first set of user-interface elements for managing one or more ephemeral event streams that contain temporarily generated time-series event data from the network packets, wherein managing the one or more ephemeral event streams comprises modifying an end time for terminating the capture of time-series event data in an ephemeral event stream. The system then updates the configuration information based on input received through the first set of user-interface elements.
H04L 41/22 - Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
H04L 43/022 - Capturing of monitoring data by sampling
H04L 43/045 - Processing captured monitoring data, e.g. for logfile generation for graphical visualisation of monitoring data
98.
Dynamic alert messages using tokens based on searching events
Custom communication alert techniques are described. In one or more implementations, a triggering condition is detected by one or more computing devices that is found by searching data using one or more extraction rules of a late-binding schema. Responsive to the detection of the triggering condition of the alert, a communication is formed by the one or more computing devices that corresponds to the alert and that includes one or more tokens based on one or more values of the data taken from fields defined by the one or more extraction rules. The communication is caused to be transmitted by the one or more computing device via a network for receipt by at least one computing device of an intended recipient of the communication.
G06F 11/07 - Responding to the occurrence of a fault, e.g. fault tolerance
H04L 41/0631 - Management of faults, events, alarms or notifications using root cause analysisManagement of faults, events, alarms or notifications using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
Described herein are technologies that facilitate effective use (e.g., indexing and searching) of non-text machine data (e.g., audio/visual data) in an event-based machine-data intake and query system.
An example method of updating a client dashboarding component of an asset monitoring and reporting system comprises: identifying an update of a client dashboarding component of an asset monitoring and reporting system (AMRS), the client dashboarding component comprising one or more dynamic elements, each dynamic element associated with an asset node; receiving one or more search queries, each search query corresponding to a dynamic element of the one or more dynamic elements; modifying one or more dynamic elements of the client dashboarding component in accordance with the one or more search queries; and updating the client dashboarding component to reflect metric values associated with the modified dynamic elements.
