diff --git a/docs/how-to/model-performance-tuning.rst b/docs/how-to/model-performance-tuning.rst new file mode 100644 index 00000000000..745abea8211 --- /dev/null +++ b/docs/how-to/model-performance-tuning.rst @@ -0,0 +1,129 @@ +.. meta:: + :description: Tune MIGraphX model performance with environment variables + :keywords: MIGraphX, performance, tuning, environment variables, ROCm + +******************************************************************** +Tune model performance +******************************************************************** + +MIGraphX exposes environment variables that change compilation behavior and +kernel selection at runtime. Set these variables before you compile or run a +model to experiment with performance options without changing application +code. + +For the complete list of environment variables, including developer-only +tracing and debugging options, see +:doc:`MIGraphX environment variables <../reference/MIGraphX-dev-env-vars>`. + +Set environment variables +==================================================================== + +Export a variable in your shell before running ``migraphx-driver`` or your +application: + +.. code-block:: shell + + export MIGRAPHX_SET_GEMM_PROVIDER=hipblaslt + /opt/rocm/bin/migraphx-driver perf model.onnx --onnx --gpu + +Replace ``model.onnx`` with your model path. + +Variables that take comma-separated lists must not include spaces between +entries. Quote the value when your shell requires it. + +Common tuning variables +==================================================================== + +The following variables are the most commonly used for model performance +tuning. Each entry describes accepted values and default behavior documented +in the MIGraphX source tree. + +.. list-table:: + :widths: 35 65 + :header-rows: 1 + + * - Environment variable + - Values + * - ``MIGRAPHX_GPU_OPTIONS`` + - JSON object of backend options (for example, ``{convolution_layout:channels_last}``). Quotes around keys and values are optional. Unrecognized options are ignored. + * - ``MIGRAPHX_SET_GEMM_PROVIDER`` + - ``hipblaslt`` or ``rocblas``. Default: ``rocblas`` on gfx90a; ``hipblaslt`` on all other architectures. + * - ``MIGRAPHX_ENABLE_GEMM_TUNING`` + - ``1`` enables exhaustive GEMM tuning even when ``--exhaustive-tune`` is not set. ``0`` returns to default behavior. + * - ``MIGRAPHX_ENABLE_HIP_GEMM_TUNING`` + - ``1`` enables exhaustive hipBLASLt tuning. ``0`` returns to default behavior. + * - ``MIGRAPHX_MLIR_TUNE_EXHAUSTIVE`` + - ``1`` enables exhaustive MLIR tuning. ``0`` returns to default behavior. + * - ``MIGRAPHX_MLIR_USE_SPECIFIC_OPS`` + - Comma-separated list of operations (for example, ``attention``, ``convolution``, ``dot``, ``fused_dot``, ``fused_convolution``, ``fused``). Prefix an entry with ``~`` to negate it. + * - ``MIGRAPHX_DISABLE_MLIR`` + - ``1`` disables rocMLIR. ``0`` returns to default behavior. + * - ``MIGRAPHX_ENABLE_CK`` + - ``1`` enables Composable Kernel. Use with ``MIGRAPHX_DISABLE_MLIR=1``. + * - ``MIGRAPHX_USE_FAST_SOFTMAX`` + - ``1`` enables fast softmax optimization. ``0`` returns to default behavior. + * - ``MIGRAPHX_ENABLE_LAYERNORM_FUSION`` + - ``1`` enables layernorm fusion. ``0`` returns to default behavior. + * - ``MIGRAPHX_FLASH_DECODING_ENABLED`` + - ``1`` enables flash decoding for attention fusion. Default: ``0``. + * - ``MIGRAPHX_SKIP_BENCHMARKING`` + - ``1`` skips kernel benchmarking and compiles with the first available solution. ``0`` returns to default behavior. + +Tuning examples +==================================================================== + +Select a GEMM provider +-------------------------------------------------------------------- + +Set the general matrix multiply (GEMM) provider before compilation: + +.. code-block:: shell + + export MIGRAPHX_SET_GEMM_PROVIDER=hipblaslt + /opt/rocm/bin/migraphx-driver perf model.onnx --onnx --gpu + +Replace ``model.onnx`` with your model path. + +Enable exhaustive GEMM tuning +-------------------------------------------------------------------- + +Search for the fastest GEMM kernel configuration: + +.. code-block:: shell + + export MIGRAPHX_ENABLE_GEMM_TUNING=1 + /opt/rocm/bin/migraphx-driver perf model.onnx --onnx --gpu --exhaustive-tune + +You can also pass ``--exhaustive-tune`` on the command line. The environment +variable enables exhaustive GEMM tuning even when that flag is omitted. + +Specify MLIR operations +-------------------------------------------------------------------- + +Force specific operations to lower through MLIR. The list is comma-separated: + +.. code-block:: shell + + export MIGRAPHX_MLIR_USE_SPECIFIC_OPS=attention,dot + /opt/rocm/bin/migraphx-driver compile model.onnx --onnx --gpu --text + +Replace ``model.onnx`` with your model path. + +Measure the effect +==================================================================== + +Compare performance before and after changing a variable: + +.. code-block:: shell + + /opt/rocm/bin/migraphx-driver perf model.onnx --onnx --gpu -n 50 + +Replace ``model.onnx`` with your model path and adjust ``-n`` for the number +of timing iterations. + +See also +==================================================================== + +* :doc:`MIGraphX driver <../migraphx-driver>` for ``perf`` and ``compile`` commands. +* :doc:`MIGraphX environment variables <../reference/MIGraphX-dev-env-vars>` for + the full variable reference, including pass controls and compilation tracing. diff --git a/docs/how-to/model-validation.rst b/docs/how-to/model-validation.rst new file mode 100644 index 00000000000..1e26b80f502 --- /dev/null +++ b/docs/how-to/model-validation.rst @@ -0,0 +1,159 @@ +.. meta:: + :description: Validate MIGraphX model outputs + :keywords: MIGraphX, validation, verify, environment variables, ROCm + +******************************************************************** +Validate model outputs +******************************************************************** + +Use MIGraphX validation tools to compare GPU or CPU target output against the +reference implementation. Validation helps confirm that compilation and +quantization changes produce numerically consistent results. + +Validate with migraphx-driver +==================================================================== + +The ``verify`` command compiles your model for the reference and target +backends, runs both, and checks that outputs match within configured +tolerances. See :doc:`MIGraphX driver <../migraphx-driver>` for full command +documentation. + +Basic verification +-------------------------------------------------------------------- + +Verify an ONNX model against the default target: + +.. code-block:: shell + + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx + +Replace ``model.onnx`` with your model path. + +Verify on GPU with tolerance settings +-------------------------------------------------------------------- + +Set absolute tolerance (``atol``), relative tolerance (``rtol``), and +root-mean-square tolerance (``rms-tol``) when comparing outputs: + +.. code-block:: shell + + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx --gpu \ + --atol 1e-5 --rtol 1e-5 --rms-tol 0.001 + +Replace ``model.onnx`` with your model path. Default tolerance values are +``0.001`` for ``atol``, ``rtol``, and ``rms-tol``. + +Verify after quantization +-------------------------------------------------------------------- + +Check that fp16 quantization preserves accuracy within your tolerances: + +.. code-block:: shell + + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx --gpu --fp16 \ + --atol 1e-3 --rtol 1e-3 + +Replace ``model.onnx`` with your model path. + +Additional verify options +-------------------------------------------------------------------- + +The driver supports these validation-related options: + +.. list-table:: + :widths: 30 70 + :header-rows: 1 + + * - Option + - Description + * - ``--per-instruction`` or ``-i`` + - Verifies each instruction individually. + * - ``--reduce`` or ``-r`` + - Reduces the program and verifies the reduced graph. + * - ``--atol`` + - Sets tolerance for elementwise absolute difference (default: ``0.001``). + * - ``--rtol`` + - Sets tolerance for elementwise relative difference (default: ``0.001``). + * - ``--rms-tol`` + - Sets tolerance for root-mean-square error (default: ``0.001``). + +Validation environment variables +==================================================================== + +Set these environment variables before running ``migraphx-driver verify`` or +your application to adjust validation behavior. + +Output comparison +-------------------------------------------------------------------- + +.. list-table:: + :widths: 35 65 + :header-rows: 1 + + * - Environment variable + - Values + * - ``MIGRAPHX_VERIFY_ENABLE_ALLCLOSE`` + - ``1`` verifies range tolerance using ``allclose``. ``0`` returns to default behavior. + * - ``MIGRAPHX_VERIFY_DUMP_DIFF`` + - ``1`` writes test output and reference output when they differ. ``0`` returns to default behavior. + +Example: + +.. code-block:: shell + + export MIGRAPHX_VERIFY_DUMP_DIFF=1 + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx --gpu + +Replace ``model.onnx`` with your model path. + +Testing and debugging +-------------------------------------------------------------------- + +These variables apply when running MIGraphX verify tests or debugging +validation failures during development: + +.. list-table:: + :widths: 35 65 + :header-rows: 1 + + * - Environment variable + - Values + * - ``MIGRAPHX_TRACE_TEST`` + - ``1`` prints reference and target programs even when verify tests pass. + * - ``MIGRAPHX_TRACE_TEST_COMPILE`` + - ``cpu`` or ``gpu`` turns on compile tracing for verify tests on the given target. Cannot be used with ``MIGRAPHX_TRACE_COMPILE``. + * - ``MIGRAPHX_DUMP_TEST`` + - ``1`` writes the model under verification to an MXR file. + +Example: + +.. code-block:: shell + + export MIGRAPHX_VERIFY_DUMP_DIFF=1 + export MIGRAPHX_TRACE_TEST=1 + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx --gpu + +Replace ``model.onnx`` with your model path. + +Graph validation during development +-------------------------------------------------------------------- + +When developing passes or matchers, enable module validation after pattern +matches: + +.. code-block:: shell + + export MIGRAPHX_VALIDATE_MATCHES=1 + +See :doc:`MIGraphX environment variables <../reference/MIGraphX-dev-env-vars>` +for the complete list of validation, testing, and tracing variables. + +See also +==================================================================== + +* :doc:`MIGraphX driver <../migraphx-driver>` for the ``verify`` command and + tolerance options. +* :doc:`Precision support <../reference/MIGraphX-data-type-support>` for + supported data types when validating quantized models. +* :doc:`Tune model performance <./model-performance-tuning>` when adjusting + compilation options that can affect numerical output. diff --git a/docs/index.rst b/docs/index.rst index d26846ffc58..8ff70d07817 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -13,17 +13,28 @@ It compiles trained models from end-to-end to optimize for inference performance The MIGraphX public repository is located at `https://github.com/ROCm/AMDMIGraphX/ `__. -You can integrate MIGraphX with PyTorch worflows by using the Torch-MIGraphX library. +You can integrate MIGraphX with PyTorch workflows by using the Torch-MIGraphX library. The public repository is located at `https://github.com/ROCm/torch_migraphx/ `__. .. grid:: 2 :gutter: 3 + .. grid-item-card:: Getting started + + * :doc:`Get started with MIGraphX <./tutorials/getting-started>` + * :doc:`Parse, load, and save a model <./tutorials/parse-load-save-tutorial>` + .. grid-item-card:: Install * :doc:`MIGraphX on ROCm installation <./install/install-migraphx>` + * :doc:`Install MIGraphX with Docker <./install/install-docker>` * :doc:`Torch-MIGraphX installation <./install/install-torch-migraphx>` + .. grid-item-card:: How-to + + * :doc:`Tune model performance <./how-to/model-performance-tuning>` + * :doc:`Validate model outputs <./how-to/model-validation>` + .. grid-item-card:: Conceptual * :doc:`Deep learning compilation with MIGraphX <./conceptual/deep-learning-compilation>` diff --git a/docs/install/install-docker.rst b/docs/install/install-docker.rst new file mode 100644 index 00000000000..45f7dd41c1d --- /dev/null +++ b/docs/install/install-docker.rst @@ -0,0 +1,106 @@ +.. meta:: + :description: Install MIGraphX using Docker + :keywords: install, Docker, MIGraphX, AMD, ROCm, container + +******************************************************************** +Install MIGraphX with Docker +******************************************************************** + +Docker provides a development environment with MIGraphX build prerequisites +preinstalled. Use this approach when you want to build MIGraphX from source +without installing dependencies on your host system. + +The default ``Dockerfile`` at the repository root builds against ROCm 7.13 and +newer using TheRock (``amdrocm-*``) packages. Alternative Dockerfiles for +other ROCm releases are available under ``tools/docker/``. + +Prerequisites +==================================================================== + +Before you build or run a MIGraphX Docker image, install the following on +your host: + +* `ROCm `__ + with GPU drivers configured +* Docker with access to ``/dev/kfd`` and ``/dev/dri`` + +Build the default Docker image +==================================================================== + +From the repository root, build the image: + +.. code-block:: shell + + docker build -t migraphx . + +To reduce image size, set ``GPU_ARCH`` to the GPU architecture on your +system. Leaving ``GPU_ARCH`` unset installs device code for all supported +architectures, which is useful when the same image runs on different +ROCm-supported GPUs. + +.. code-block:: shell + + docker build -t migraphx --build-arg GPU_ARCH=$(rocminfo | grep -o -m1 'gfx.*') . + +The root ``Dockerfile`` accepts the following build arguments: + +.. list-table:: + :widths: 30 70 + :header-rows: 1 + + * - Build argument + - Description + * - ``ROCM_VERSION`` + - ROCm release version for versioned package names (default: ``7.14``). + * - ``GPU_ARCH`` + - GPU architecture family (for example, ``gfx942``, ``gfx120x``). Leave empty for arch-independent packages. + * - ``USE_WHL`` + - When set to a non-empty value, installs ROCm from Python wheels instead of system packages. + * - ``INDEX_URL`` + - pip index URL used when ``USE_WHL`` is set (default: ``https://repo.amd.com/rocm/whl-multi-arch/``). + +Run the container +==================================================================== + +Start an interactive container with your repository mounted at +``/code/AMDMIGraphX``: + +.. code-block:: shell + + docker run --device='/dev/kfd' --device='/dev/dri' \ + -v=`pwd`:/code/AMDMIGraphX -w /code/AMDMIGraphX \ + --group-add video -it migraphx + +Build MIGraphX inside the container +==================================================================== + +Inside the container, prerequisites are already installed. Follow the steps in +:doc:`MIGraphX on ROCm installation <./install-migraphx>` to build from source, +starting from the rbuild or CMake build steps. + +Alternative Dockerfiles +==================================================================== + +Use an alternative Dockerfile when your target ROCm release requires a +different base image or package source. + +.. list-table:: + :widths: 35 65 + :header-rows: 1 + + * - Dockerfile path + - Use when + * - ``tools/docker/legacy.dockerfile`` + - Building against ROCm 7.2.x and older on Ubuntu 22.04. + * - ``tools/docker/ubuntu_2404.dockerfile`` + - Building on Ubuntu 24.04 with ROCm 7.1.1. + * - ``tools/docker/ubuntu_2204.dockerfile`` + - Building on Ubuntu 22.04 with ROCm 6.4.2. + +Build with the ``-f`` flag to select a Dockerfile. For example: + +.. code-block:: shell + + docker build -t migraphx:legacy -f tools/docker/legacy.dockerfile . + +Then follow the same ``docker run`` and build steps described above. diff --git a/docs/install/install-migraphx.rst b/docs/install/install-migraphx.rst index 3eaaa4ed64b..f206c8e0542 100644 --- a/docs/install/install-migraphx.rst +++ b/docs/install/install-migraphx.rst @@ -12,7 +12,10 @@ for instructions. Installing MIGraphX using a package installer is sufficient for most users who want to use the MIGraphX API. If you plan to develop for MIGraphX or -contribute to the source code, see `Developing for MIGraphX <../dev/contributing-to-migraphx.html>`_ +contribute to the source code, see `Developing for MIGraphX <../dev/contributing-to-migraphx.html>`_. + +For a containerized build environment with prerequisites preinstalled, see +:doc:`Install MIGraphX with Docker <./install-docker>`. Install MIGraphX with a package installer ==================================================================== diff --git a/docs/reference/MIGraphX-dev-env-vars.rst b/docs/reference/MIGraphX-dev-env-vars.rst index 746f5b62618..b5e4e3e9b45 100644 --- a/docs/reference/MIGraphX-dev-env-vars.rst +++ b/docs/reference/MIGraphX-dev-env-vars.rst @@ -8,6 +8,10 @@ MIGraphX environment variables The MIGraphX environment variables can be used by contributors to the MIGraphX code base to customize tuning, verification, and tracing. +For user-facing guides on the most common tuning and validation variables, see +:doc:`Tune model performance <../how-to/model-performance-tuning>` and +:doc:`Validate model outputs <../how-to/model-validation>`. + Model performance tunable variables ************************************ diff --git a/docs/reference/MIGraphX-user-reference.rst b/docs/reference/MIGraphX-user-reference.rst index f88208bdc0e..69e3f178ff7 100644 --- a/docs/reference/MIGraphX-user-reference.rst +++ b/docs/reference/MIGraphX-user-reference.rst @@ -6,6 +6,12 @@ MIGraphX user reference ======================================================== -The MIGraphX library includes :doc:`C++ <./MIGraphX-cpp>` and :doc:`Python <./MIGraphX-py>` APIs for creating applications. +The MIGraphX library includes :doc:`C++ <./MIGraphX-cpp>` and :doc:`Python <./MIGraphX-py>` APIs for creating applications. + +Operational guides: + +* :doc:`Tune model performance <../how-to/model-performance-tuning>` +* :doc:`Validate model outputs <../how-to/model-validation>` +* :doc:`MIGraphX driver <../migraphx-driver>` To contribute to the MIGraphX code base, see :doc:`the MIGraphX contributor documentation <./MIGraphX-dev-reference>`. \ No newline at end of file diff --git a/docs/sphinx/_toc.yml.in b/docs/sphinx/_toc.yml.in index 79e4210f8a6..78b4912e350 100644 --- a/docs/sphinx/_toc.yml.in +++ b/docs/sphinx/_toc.yml.in @@ -3,9 +3,18 @@ root: index subtrees: + - caption: Getting started + entries: + - file: tutorials/getting-started + title: Get started with MIGraphX + - file: tutorials/parse-load-save-tutorial + title: Parse, load, and save a model + - caption: Installation entries: - file: install/install-migraphx + - file: install/install-docker + title: Install MIGraphX with Docker - file: install/install-torch-migraphx - caption: Conceptual @@ -13,6 +22,13 @@ subtrees: - file: conceptual/deep-learning-compilation title: Deep learning compilation with MIGraphX + - caption: How-to + entries: + - file: how-to/model-performance-tuning + title: Tune model performance + - file: how-to/model-validation + title: Validate model outputs + - caption: Reference entries: - file: reference/MIGraphX-user-reference diff --git a/docs/tutorials/MIGraphX-examples.rst b/docs/tutorials/MIGraphX-examples.rst index f93f86ce8e4..99570f87b83 100644 --- a/docs/tutorials/MIGraphX-examples.rst +++ b/docs/tutorials/MIGraphX-examples.rst @@ -1,16 +1,204 @@ .. meta:: - :description: MIGraphX Examples + :description: MIGraphX examples :keywords: MIGraphX, AMD, ROCm, examples ******************************************************************** MIGraphX examples ******************************************************************** -Example code for the following use cases is available in the `MIGraphX GitHub repository `_: +The MIGraphX repository includes examples for common inference workflows. +Each example below is documented in its README file. Follow the linked path +in the `MIGraphX GitHub repository `__ +to build and run the example. -* `Diffusion inference `_ -* `AMD MIGraphX usage and utilities `_ -* `Natural language processing inference `_ -* `ONNX runtime `_ -* `Transformer inference `_ -* `Vision inference `_ \ No newline at end of file +To swap in your own model, replace the model file path named in the example +README or source code with your ONNX or serialized MIGraphX program. Adjust +input dimensions and data loading to match your model's parameter shapes. + +Getting started examples +==================================================================== + +These examples teach core MIGraphX workflows. Start here if you are new to +the library. + +.. list-table:: + :widths: 25 35 40 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - :doc:`Parse, load, and save a model <./parse-load-save-tutorial>` + - C++ application developers + - Parse ONNX models, inspect graphs, and save or load programs in MessagePack or JSON format. + * - `cpp_mnist `__ + - C++ application developers + - End-to-end inference: parse ONNX, compile for GPU or CPU, quantize, prepare inputs, and evaluate. + * - `migraphx_driver `__ + - Model integrators + - Command-line usage of ``migraphx-driver`` for read, compile, run, verify, and perf without writing code. + +MIGraphX API examples +==================================================================== + +Examples under ``examples/migraphx/`` cover MIGraphX-specific utilities and +advanced C++ API usage. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `cpp_parse_load_save `__ + - C++ application developers + - Parse, load, and save graph programs. Swap in your ONNX file as the input argument. + * - `cpp_dynamic_batch `__ + - C++ application developers + - Run programs with dynamic batch sizes using ``dynamic_dimension`` objects. Replace the ONNX input file with your dynamic-batch model. + * - `cpp_trace_callback `__ + - MIGraphX contributors and debuggers + - Inspect operator output buffers during evaluation with ``program::run_trace``. + * - `export_frozen_graph_tf1 `__ + - Model integrators + - Export frozen TensorFlow 1 graphs for MIGraphX ingestion. + * - `export_frozen_graph_tf2 `__ + - Model integrators + - Export frozen TensorFlow 2 graphs for MIGraphX ingestion. + +Vision examples +==================================================================== + +Examples under ``examples/vision/`` cover image classification, detection, +segmentation, and super resolution. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `cpp_mnist `__ + - C++ application developers + - MNIST digit classification with the C++ API. + * - `python_resnet50 `__ + - Python application developers + - ResNet50 V2 inference in Python via a Jupyter notebook. Replace the bundled model with your own pre-trained classification model. + * - `python_nfnet `__ + - Python application developers + - NFNet inference and ONNX Runtime comparison. + * - `python_super_resolution `__ + - Python application developers + - Super resolution inference. + * - `python_unet `__ + - Python application developers + - U-Net segmentation inference. + * - `python_3dunet `__ + - Python application developers + - 3D U-Net inference. + * - `python_yolov4 `__ + - Python application developers + - YOLOv4 object detection inference. + * - `python_yolo26 `__ + - Python application developers + - YOLO26 object detection inference. + +Natural language processing examples +==================================================================== + +Examples under ``examples/nlp/`` cover question answering and speech +recognition. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `python_bert_squad `__ + - Python application developers + - BERT question answering on SQuAD. Replace ``bertsquad-10.onnx`` and ``inputs.json`` with your model and input data. + * - `python_rnnt `__ + - Python application developers + - RNN-T speech recognition inference. + +Diffusion examples +==================================================================== + +Examples under ``examples/diffusion/`` cover text-to-image diffusion models. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `python_stable_diffusion_21 `__ + - Python application developers + - Stable Diffusion 2.1 text-to-image inference. + * - `python_stable_diffusion_xl `__ + - Python application developers + - Stable Diffusion XL text-to-image inference. + * - `python_stable_diffusion_3 `__ + - Python application developers + - Stable Diffusion 3 text-to-image inference. + * - `python_flux `__ + - Python application developers + - Flux text-to-image inference. + * - `python_controlnet_canny_sd_15 `__ + - Python application developers + - ControlNet Canny edge guidance with Stable Diffusion 1.5. + +Transformer examples +==================================================================== + +Examples under ``examples/transformers/`` cover large language model and +speech model inference. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `python_llama2 `__ + - Python application developers + - Llama 2 inference. + * - `python_whisper `__ + - Python application developers + - Whisper automatic speech recognition. + +ONNX Runtime examples +==================================================================== + +Examples under ``examples/onnxruntime/`` show MIGraphX integration through +the ONNX Runtime execution provider. + +.. list-table:: + :widths: 30 35 35 + :header-rows: 1 + + * - Example + - Target user + - What it demonstrates + * - `resnet50 `__ + - ONNX Runtime users + - ResNet50 inference through ONNX Runtime with the MIGraphX execution provider. + * - `inceptionV3 `__ + - ONNX Runtime users + - InceptionV3 inference through ONNX Runtime with the MIGraphX execution provider. + +See also +==================================================================== + +* :doc:`Get started with MIGraphX <./getting-started>` for persona-based + starting paths. +* :doc:`Install MIGraphX with Docker <../install/install-docker>` for a + containerized build environment. +* :doc:`Validate model outputs <../how-to/model-validation>` to check example + output against the reference implementation. diff --git a/docs/tutorials/getting-started.rst b/docs/tutorials/getting-started.rst new file mode 100644 index 00000000000..e4a242e6c41 --- /dev/null +++ b/docs/tutorials/getting-started.rst @@ -0,0 +1,132 @@ +.. meta:: + :description: Get started with MIGraphX after installation + :keywords: MIGraphX, getting started, tutorial, ROCm + +******************************************************************** +Get started with MIGraphX +******************************************************************** + +After you install MIGraphX, use this guide to choose a starting path based on +how you plan to use the library. Each section links to validated examples and +reference material in this repository. + +Choose your path +==================================================================== + +MIGraphX supports several integration patterns. Pick the section that matches +your role and goal. + +Application developer using the C++ API +==================================================================== + +Use this path when you embed MIGraphX directly in a C++ application. + +1. Confirm MIGraphX is installed. See :doc:`MIGraphX on ROCm installation <../install/install-migraphx>`. +2. Set ``CMAKE_PREFIX_PATH`` to your MIGraphX installation location and link + against the C++ API: + + .. code-block:: cmake + + find_package(migraphx) + target_link_libraries(my_app migraphx::c) + + Replace ``my_app`` with your CMake target name. +3. Work through :doc:`Parse, load, and save a model <./parse-load-save-tutorial>` + to learn how to load ONNX models and serialize programs. +4. Continue with the C++ MNIST inference example at + ``examples/vision/cpp_mnist/`` in the + `MIGraphX GitHub repository `__ + for a complete parse, compile, and evaluate workflow. +5. Consult the :ref:`cpp-api-reference` for API details. + +Application developer using the Python API +==================================================================== + +Use this path when you call MIGraphX from Python. + +1. Confirm MIGraphX is installed. See :doc:`MIGraphX on ROCm installation <../install/install-migraphx>`. +2. Add the Python module to your environment: + + .. code-block:: shell + + export PYTHONPATH=/opt/rocm/lib:$PYTHONPATH + + Replace ``/opt/rocm/lib`` with the directory that contains the MIGraphX + Python module if your installation path differs. +3. Browse the Python examples under ``examples/`` in the + `MIGraphX GitHub repository `__. + Start with a domain that matches your model, such as vision or natural + language processing. +4. Consult the :ref:`python-api-reference` for API details. + +Model integrator or inference engineer +==================================================================== + +Use this path when you want to inspect, compile, verify, or benchmark models +without writing application code. + +1. Confirm ``migraphx-driver`` is available at ``/opt/rocm/bin/migraphx-driver``, + or at ``AMDMIGraphX/build/bin/migraphx-driver`` after a source build. +2. Read :doc:`MIGraphX driver <../migraphx-driver>` for commands such as + ``read``, ``compile``, ``run``, ``verify``, and ``perf``. +3. Validate a model against the reference implementation: + + .. code-block:: shell + + /opt/rocm/bin/migraphx-driver verify model.onnx --onnx + +4. Measure performance: + + .. code-block:: shell + + /opt/rocm/bin/migraphx-driver perf model.onnx --onnx --gpu -n 50 + +5. See :doc:`Validate model outputs <../how-to/model-validation>` and + :doc:`Tune model performance <../how-to/model-performance-tuning>` for + environment variables and tolerance settings. + +PyTorch user +==================================================================== + +Use this path when you integrate MIGraphX with PyTorch workflows through +Torch-MIGraphX. + +1. Install Torch-MIGraphX. See + :doc:`Torch-MIGraphX installation <../install/install-torch-migraphx>`. +2. Refer to the Torch-MIGraphX repository at + `https://github.com/ROCm/torch_migraphx/ `__ + for model conversion and inference examples. + +ONNX Runtime user +==================================================================== + +Use this path when you run inference through the ONNX Runtime MIGraphX +execution provider. + +1. Install MIGraphX. See :doc:`MIGraphX on ROCm installation <../install/install-migraphx>`. +2. Follow the ResNet50 example at + ``examples/onnxruntime/resnet50/`` in the + `MIGraphX GitHub repository `__ + for setup and inference steps with ONNX Runtime. + +MIGraphX contributor +==================================================================== + +Use this path when you modify the MIGraphX source code or add operators and +passes. + +1. Set up a build environment. See :doc:`Install MIGraphX with Docker <../install/install-docker>` + or :doc:`MIGraphX on ROCm installation <../install/install-migraphx>`. +2. Read :doc:`Develop for the MIGraphX code base <../dev/contributing-to-migraphx>`. +3. Consult :doc:`MIGraphX environment variables <../reference/MIGraphX-dev-env-vars>` + for tuning, tracing, and testing options used during development. + +Next steps +==================================================================== + +* :doc:`Parse, load, and save a model <./parse-load-save-tutorial>` walks + through your first C++ workflow with an ONNX model. +* :doc:`MIGraphX examples <./MIGraphX-examples>` lists documented examples by + domain and target user. +* :doc:`Deep learning compilation with MIGraphX <../conceptual/deep-learning-compilation>` + explains how MIGraphX optimizes inference graphs. diff --git a/docs/tutorials/parse-load-save-tutorial.rst b/docs/tutorials/parse-load-save-tutorial.rst new file mode 100644 index 00000000000..b68e9389e0f --- /dev/null +++ b/docs/tutorials/parse-load-save-tutorial.rst @@ -0,0 +1,131 @@ +.. meta:: + :description: Parse, load, and save MIGraphX programs + :keywords: MIGraphX, tutorial, ONNX, parse, load, save, C++ + +******************************************************************** +Parse, load, and save a model +******************************************************************** + +This tutorial walks through the +``examples/migraphx/cpp_parse_load_save`` example in the MIGraphX repository. +You learn how to parse an ONNX model into a MIGraphX program, inspect the +graph, save it to MessagePack format, and reload it later. + +The example source is at +`examples/migraphx/cpp_parse_load_save `__ +in the MIGraphX GitHub repository. + +What you will learn +==================================================================== + +* Parse an ONNX file into a ``migraphx::program``. +* Save a program to MessagePack (``.mxr``) format. +* Load a previously saved program from MessagePack or JSON format. +* Swap in your own ONNX model by changing the input file path. + +Prerequisites +==================================================================== + +* MIGraphX installed. See :doc:`MIGraphX on ROCm installation <../install/install-migraphx>`. +* A C++ compiler and CMake 3.5 or later. +* An ONNX model file. The tutorial uses any compatible ONNX file you provide + as ``input_model.onnx``. + +Build the example +==================================================================== + +From the ``examples/migraphx/cpp_parse_load_save`` directory: + +.. code-block:: shell + + mkdir build + cd build + cmake .. + make + +This produces an executable named ``parse_load_save``. + +Parse an ONNX model +==================================================================== + +By default, the program parses the input file as ONNX and prints the internal +graph structure. Replace ``input_model.onnx`` with the path to your model: + +.. code-block:: shell + + ./parse_load_save input_model.onnx --parse onnx + +The C++ API call used in the example is: + +.. code-block:: cpp + + migraphx::program p; + migraphx::onnx_options options; + p = parse_onnx(input_file, options); + +Computation graphs saved in a compatible serialized format, such as ONNX, can +be read by MIGraphX to create a runnable program. + +Save a program +==================================================================== + +Save the parsed program to MessagePack format. Replace ``output_name`` with +your preferred base filename (the ``.mxr`` extension is added automatically): + +.. code-block:: shell + + ./parse_load_save input_model.onnx --parse onnx --save output_name + +The program writes ``output_name.mxr`` in the current directory. + +Load a saved program +==================================================================== + +Load a program that was previously saved in MessagePack format: + +.. code-block:: shell + + ./parse_load_save output_name.mxr --load msgpack + +To load a program saved in JSON format: + +.. code-block:: shell + + ./parse_load_save saved_program.json --load json + +Use your own model and data +==================================================================== + +To adapt this example for your workflow: + +1. Replace ``input_model.onnx`` with the path to your ONNX model file. +2. If your model has dynamic or unspecified dimensions, configure + ``migraphx::onnx_options`` before parsing. For example, set a default batch + size with ``options.set_default_dim_value(batch)``, where ``batch`` is your + batch size integer. +3. After parsing, compile the program for your target and run inference. See + the C++ MNIST example at + ``examples/vision/cpp_mnist/`` for compile and evaluate steps. + +Command-line reference +==================================================================== + +The ``parse_load_save`` executable supports the following usage: + +.. code-block:: text + + ./parse_load_save [options] + options: + --parse onnx + --load json/msgpack + --save + +Next steps +==================================================================== + +* Complete an end-to-end inference workflow in the C++ MNIST example + documented at ``examples/vision/cpp_mnist/``. +* Use :doc:`MIGraphX driver <../migraphx-driver>` to inspect and verify your + model from the command line without writing C++ code. +* See :doc:`MIGraphX examples <./MIGraphX-examples>` for additional examples + by domain. diff --git a/examples/migraphx/migraphx_docker/README.md b/examples/migraphx/migraphx_docker/README.md index 55571a42b88..3a1e4830e71 100644 --- a/examples/migraphx/migraphx_docker/README.md +++ b/examples/migraphx/migraphx_docker/README.md @@ -1,3 +1,6 @@ # MIGraphX Dockerfile -Instructions for building and running the MIGraphX docker container can be found [here](https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/blob/develop/README.md#using-docker) in this project's top level README. +Instructions for building and running the MIGraphX Docker container are in the +[MIGraphX documentation](https://rocm.docs.amd.com/projects/AMDMIGraphX/en/latest/install/install-docker.html) +and in the [Use Docker](https://github.com/ROCm/AMDMIGraphX/blob/develop/README.md#use-docker) +section of the repository README.