Skip to content
forked from Texera/texera

Collaborative Machine-Learning-Centric Data Analytics Using Workflows

Notifications You must be signed in to change notification settings

sreetej1998/texera

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Texera - Collaborative Data Analytics Using Workflows.

texera-logo
Texera supports scalable computation and enables advanced AI/ML techniques.
"Collaboration" is a key focus, and we enable an experience similar to Google Docs, but for data analytics.

Motivation

  • Many data analysts need to spend a significant amount of effort on low-level computation to do data wrangling and preparation, and want to use latest AI/ML techniques. These tasks are especially tough for non-IT users.

  • Many workflow-based analysis systems are not parallel, making them not capable of dealing with big data sets.

  • Cloud-based services and technologies have emerged and advanced significantly in the past decade. Emerging browser-based techniques make it possible to develop powerful browser-based interfaces, which also benefit from high-speed networks.

  • Existing big data systems support little interaction during the execution of a long running job, making them hard to manage once they are started.

Goals

  • Provide data analytics as cloud services;
  • Provide a browser-based GUI to form a workflow without writing code;
  • Allow non-IT people to do data analytics;
  • Support collaborative data analytics;
  • Allow users to interact with the execution of a job;
  • Support huge volumes of data efficiently.

Sample Workflow

The following is a workflow formulated using the Texera GUI in a Web browser, which consists of operators such as regex search, sentiment analysis, user-defined function (UDF) in Python, and visualization.

Sample Texera Workflow

Publications (Computer Science):

  • (4/2017) A Demonstration of TextDB: Declarative and Scalable Text Analytics on Large Data Sets, Zuozhi Wang, Flavio Bayer, Seungjin Lee, Kishore Narendran, Xuxi Pan, Qing Tang, Jimmy Wang, Chen Li, ICDE 2017, Best Demo award, PDF, Video.
  • (1/2020) Amber: A Debuggable Dataflow system based on the Actor Model, Avinash Kumar, Zuozhi Wang, Shengquan Ni, Chen Li, VLDB 2020 PDF, Video, Slides
  • (7/2020) Demonstration of Interactive Runtime Debugging of Distributed Dataflows in Texera, Zuozhi Wang, Avinash Kumar, Shengquan Ni, Chen Li, VLDB 2020 PDF, Video, Slides
  • (4/2022) Optimizing Machine Learning Inference Queries with Correlative Proxy Models, Zhihui Yang, Zuozhi Wang, Yicong Huang, Yao Lu, Chen Li, X. Sean Wang, VLDB 2022 PDF.
  • (6/2022) Demonstration of Collaborative and Interactive Workflow-Based Data Analytics in Texera, Xiaozhen Liu, Zuozhi Wang, Shengquan Ni, Sadeem Alsudais, Yicong Huang, Avinash Kumar, Chen Li, in VLDB 2022 PDF, Demo Video.
  • (6/2022) Demonstration of Accelerating Machine Learning Inference Queries with Correlative Proxy Models, Zhihui Yang, Yicong Huang, Zuozhi Wang, Feng Gao, Yao Lu, Chen Li, X. Sean Wang, in VLDB 2022 PDF .
  • (7/2022) Drove: Tracking Execution Results of Workflows on Large Datasets, Sadeem Alsudais, in the PhD workshop at VLDB 2022 PDF.
  • (9/2022) Fries: Fast and Consistent Runtime Reconfiguration in Dataflow Systems with Transactional Guarantees, Zuozhi Wang, Shengquan Ni, Avinash Kumar, Chen Li, VLDB 2023 PDF.
  • (12/2022) Towards Interactive, Adaptive and Result-aware Big Data Analytics, Avinash Kumar, Phd Thesis PDF.

Publications (Interdisciplinary):

  • (4/2021) Why Do People Oppose Mask Wearing? A Comprehensive Analysis of US Tweets During the COVID-19 Pandemic, Lu He, Changyang He, Tera Leigh Reynolds, Qiushi Bai, Yicong Huang, Chen Li, Kai Zheng, and Yunan Chen in JAMIA 2021 PDF.
  • (9/2021) The Social Amplification and Attenuation of COVID-19 Risk Perception Shaping Mask Wearing Behavior: A Longitudinal Twitter Analysis, Suellen Hopfer, Emilia J. Fields, Yuwen Lu, Ganesh Ramakrishnan, Ted Grover, Quishi Bai, Yicong Huang, Chen Li, and Gloria Mark in PLOS ONE, 2021 PDF.

Videos

Getting Started

Texera was formally known as "TextDB" before August 28, 2017.

Instructions for VLDB 2022 Demo Paper

To try our collaborative data analytics in Demonstration of Collaborative and Interactive Workflow-Based Data Analytics in Texera, visit https://github.com/Texera/texera/wiki/Instructions-for-VLDB-2022-Demo.

Acknowledgements

This project is supported by the National Science Foundation under the awards III 1745673, III 2107150, AWS Research Credits, and Google Cloud Platform Education Programs.

  • Yourkit Yourkit has given an open source license to use their profiler in this project.

About

Collaborative Machine-Learning-Centric Data Analytics Using Workflows

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Scala 31.0%
  • Java 29.7%
  • TypeScript 21.5%
  • HTML 12.3%
  • Python 4.1%
  • SCSS 0.7%
  • Other 0.7%