Large-scale adaptive modeling in real-time

Flowdapt is an Emergent Methods project for large-scale adaptive modeling challenges. It is highly adaptable to handle customized workflows (plug-ins) in parallelized environments. And the best part, is that Flowdapt is fully open-source

Distributed compute

Customizable plug-ins

Flowdapt is built to adopt any application that may require high-level orchestration of compute-heavy libraries (i.e. PyTorch) as well as cluster microservices. Some of the most notable plug-ins include:

• •
  AskNews: a contextualized, on-premise, user-specific LLM deployment based on Llama2. AskNews summarizes and indexes all the latest news so that you can ask natural language questions and get and up-to-date distillation of information with hyperlinks and article titles to all the related news sources.
• •
  Nowcast: a weather forecasting plug-in designed to provide short-term hyper-local forecasts for hundreds of cities simultaneously. The plug-in collects new meteorological data as it becomes avaialable and retrains models if their performance "drifts."
• •
  CryptoCast: a cryptocurrency market forecast plug-in designed to provide a highly modifiable interface for per-asset model building/updating.
• •
  AskArxiv: a contextualized, on-premise LLM deployment that summarizes and indexes all the latest ArXiv articles. Users can ask natural language questions about the latest academic breakthroughs.

Real-time adaptive modeling for finance

FreqAI is an open-source project that allows for time-series forecasting via established machine learning algorithms, such as Reinforcement Learning, Decision Trees, and Neural Networks.
Built in Python, the core engine allows users to connect any open-source machine learning or data analysis library to create a custom system for adaptive modeling. We currently provide 18 pre-configured prediction models for XGBoost, LightGBM, CatBoost, TensorFlow, PyTorch, and Stable Baselines.

DISCLAIMER FreqAI has gained such a strong reputation in the open-source machine learning community that it has become a target of imposter projects aimed at using the FreqAI name to offer cryptocurrencies. FreqAI is not affiliated with any cryptocurrency offerings. FreqAI is, and always will be, a not-for-profit, open-source project. FreqAI does not have a crypto token, FreqAI does not sell signals, and FreqAI does not have a domain besides the freqtrade documentation https://www.freqtrade.io/en/latest/freqai/. Please beware of imposter projects, and help us by reporting them to the official FreqAI discord server.

A Python library for complex data pipelines

DataSieve is an Emergent Methods tool designed to add key features to the SKLearn pipeline, including:

• • A variety of custom transforms (Dissimilarity Index, SVM outlier extractor, etc.)
• • Outlier removal across your X, y, and sample weights arrays according to simple or complex criteria.
• • Feature column removal based on arbitrary criteria (e.g., low variance features.)
• • Feature column name changes for certain transformations (e.g., PCA.)
• • Outlier classification without removal.
• • Backend customization for parallelization (e.g., Dask, Ray, loky, etc.)

DataSieve is deployed in a variety of adaptive-modeling softwares such as FreqAI and Flowdapt.

Reducing friction between configuration and code

Manifest is an Emergent Methods tool designed to improve the flexibility of the python software configuration process. Manifest allows you to:

• • Define custom models to validate your configurations.
• • Load configurations from a variety of sources, such as environment variables, JSON files, YAML files, or TOML files.
• • Instantiate Python objects directly from your configurations.
• • Leverage dynamic configurations through expressions.
• • Easily serialize and deserialize Manifests based on file extensions.

Manifest is deployed in Flowdapt.

An LLM abstraction library providing high-level objects for complex thought strategies

DeepThought is an Emergent Methods library designed to help us easily manipulate high-level LLM abstractions. The library is in active development and it includes:

• • Industry standard Tree of Thought, Chain of Thought, and Meta Prompting strategies.
• • Experimental thought strategies.
• • High-level abstractions on long-term/short-term memory adaptation.
• • Adding Flowdapt workflow execution capabilities to LLM thought processes.

A library geared toward efficiently scaling applications that need consensus algorithms.

Adrift is an Emergent Methods tool designed to more efficiently scale duplicate services in need of consensus. The library is in active development and it includes:

• • Industry standard consensus algorithms.
• • Experimental consensus algorithms geared toward improved efficiency.