About Project

MBASE inference library is a high-level non-blocking LLM inference library written on top of the llama.cpp library to provide the necessary tools and APIs to allow developers to integrate popular LLMs into their applications with minimal performance loss and development time.

Features can be listed as:

• Non-blocking TextToText LLM inference SDK.
• Non-blocking Embedder model inference SDK.
• GGUF file meta-data manipulation SDK.
• Openai server program supporting both TextToText and Embedder endpoints with system prompt caching support which implies significant performance boost.
• Hosting multiple models in a single Openai server program.
• Using llama.cpp as an inference backend so that models that are supported by the llama.cpp library are supported by default.
• Benchmark application for measuring the impact of LLM inference on your application.
• Plus anything llama.cpp supports.

For more detail, go to the github repository

For documentation, go to the documentation

About Company

MBASE Yazılım A.Ş. (MBASE Software Corporation) is a software company in Turkey/Ankara founded by Mustafa Emre Erdoğ (Using Saul Emre Erdoğ as a preferred name).

The company is being established to accelerate the development of the MBASE SDK and for creating both proprietary and open-source products using the MBASE SDK.

The goal of the MBASE SDK is expected to provide non-blocking LLM inference to both beginner and advanced users of the C++ library and very useful tools and programs to non-programmer users.

Repository

MBASE Project is an open-source project, actively developed at github repository. We welcome and appreciate contributions from the community.

Contact

For any business inquiries, consultancy, deployment, partnership or, professional contribution to the MBASE project, send us an email at [email protected]