ppl.pmx is(but not only) a part of PPL.LLM system.
We recommend users who are new to this project to read the Overview of system.
PPL Model Exchange (PMX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. PMX provides an open source format for AI models. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.
Currently PMX focus on the capabilities and hardware friendliness needed for Large Language Model(LLM) inferencing.
Table of Contents: Link
About add new operator: Link
About update an operator's version: Link
PMX provides functional API based on torch.autograd.Function.
Clone the PMX repo, and import torch_function like this:
import pmx_llm.torch_function as PMXAnd then use it as Pytorch's functional API:
norm, skip_out = PMX.skip_rms_norm(x, weight, skip_in, -1, eps)We can use these API in pytorch to custom your own model.
All PMX function could be exported as custom operators by torch.onnx.export.
Some opensource model are provided in our model zoo.
Currently models: