Accelerated analytics and data science at massive scale

OmniSci users will get deeper insights interactively on multi-billion row datasets with newly announced Data Fusion and Data Science capabilities.

Wednesday, 23rd October 2019 Posted 5 years ago in by Phil Alsop

OmniSci has introduced OmniSci 5.0, a major step forward in making analytics instant, powerful and effortless for everyone available later this year. Users will be able to quickly and easily augment their existing datasets in OmniSci, with external data from partners and public data sources, providing greater analytical breadth and perspective. Through deep integration with the open source data science ecosystem, OmniSci offers users the ability to seamlessly switch from interactive exploration of multi-billion row datasets, to building and applying Machine Learning (ML) models on this data for greater depth of insight, all with unparalleled speed and agility.

“At OmniSci we are working to converge analytics, data science and location intelligence, at scale, into one seamless workflow, to help answer the biggest questions at the speed of curiosity,” said Venkat Krishnamurthy, OmniSci vice president of product management. “OmniSci 5.0 represents a major step towards that goal. The new Data Fusion and integrated Data Science capabilities, along with significant performance improvements in the core platform, offer our users, whether they are data scientists, business analysts or geospatial analysts, the ability to quickly fuse multiple perspectives from both their own data and other relevant datasets, and switch between visual exploration and data science workflows seamlessly to extract the deepest insights possible.”

Key features in OmniSci 5.0 Immerse include:

An integrated Data Catalogue with the import external public or partner datasets and visually join them in geospatial or time series charts in Immerse.
Visual analytics at the entity level, including the ability to interactively build and analyse the behaviour of cohorts in data (including people and moving vehicles such as ships, planes and cars).

The integrated Data Science capabilities in OmniSci 5.0 allow data scientists to switch between visual analytics in Immerse, to deeper exploration of the same data with Machine Learning. Key features in this regard include:

Integration with the PyData stack lets OmniSci users use Immerse and JupyterLab, the next-generation notebook interface from the Jupyter project, in a single workflow to select a subset of data in Immerse and then launch a notebook to explore this data further to build ML models.
OmniSciDB 5.0 adds foundational support for User Defined Functions both at the row and table level, to allow external ML libraries to be integrated into query execution, both to train and apply ML models and consume their results in Immerse.

OmniSciDB also includes support for quicker export and restore capabilities via a new compressed binary format, as well as a Foreign Storage Interface (in beta) feature that allows OmniSci to attach to other data stores and analyse data, without needing to import and store that data. Additionally, foundational work on performance and scalability has resulted in 8x improvement in performance for certain aggregate queries, as well as an improved architecture for resiliency and high availability.

“We have been partners with OmniSci since 2018, working with the Open Data Science community on key projects in the PyData ecosystem targeting use cases in Interactive Analytics and Machine Learning at scale - including Altair, JupyterLab, Ibis and Vega. This type of industry collaboration is a model for how companies can both leverage the power of these communities and their contributors, while contributing back to them in a sustainable way,” said Travis Oliphant, CEO of Quansight Labs, founder of OpenTeams and Anaconda, and creator of the NumPy, Numba and SciPy projects. “We look forward to continued collaboration around our shared goal to make modern data science more accessible and performant at scale.”