50 Pull requests merged by 27 people

• pick changes from https://github.com/onnx/onnx/pull/6195 to fix heap-buffer-overflow in onnx::convPoolShapeInference

  #21507 merged Jul 27, 2024

• Delete tools/ci_build/github/azure-pipelines/win-gpu-ci-pipeline.yml

  #21529 merged Jul 27, 2024

• [AIX]test failure fix using gtest-1.15.0 for AIX

  #21497 merged Jul 27, 2024

• Security fuzz address sanitizer fix Bug #2 and #3

  #21528 merged Jul 27, 2024

• Bump Sixlabors.ImageSharp from 2.1.8 to 2.1.9 in /csharp/sample/Microsoft.ML.OnnxRuntime.ResNet50v2Sample

  #21444 merged Jul 27, 2024

• Fix conda failure for onnxruntime-directml

  #21526 merged Jul 27, 2024

• [VitisAI] support vaip create ep context nodes & bug fix

  #21506 merged Jul 27, 2024

• [VitisAI] 1. KernelDef supports StartVersion and EndVersion

  #21519 merged Jul 27, 2024

• Set version and other info in the C# dll

  #21517 merged Jul 27, 2024

• Update benchmark_mha.py to compare with PyTorch SDPA

  #21449 merged Jul 27, 2024

• Add QNN EP option context_node_name_prefix to set EPContext node name prefix

  #21236 merged Jul 26, 2024

• Separating all GPU stages into different Pipelines

  #21521 merged Jul 26, 2024

• Potential fix for Adobe Analytics

  #21512 merged Jul 26, 2024

• Update text formatting in generate_cgmanifest.py

  #21489 merged Jul 26, 2024

• disables qnn in ort training cpu pipeline

  #21510 merged Jul 26, 2024

• [WebNN EP] Update argMax/argMin to adapt to latest spec

  #21452 merged Jul 26, 2024

• [DML EP] Register ReduceMin-20

  #20477 merged Jul 26, 2024

• Fix SkipLayerNormFusion incorrectly setting modified every time it runs

  #21502 merged Jul 26, 2024

• Allow cpplint to always be green

  #21491 merged Jul 25, 2024

• CoreML: Aggregated changes to add all required ops for priority model

  #21472 merged Jul 25, 2024

• Fix Android CI Pipeline code coverage failure

  #21504 merged Jul 25, 2024

• Qnn batchnorm support input with rank 2

  #21469 merged Jul 25, 2024

• Add community-contributed LightGlue blog

  #21445 merged Jul 25, 2024

• Split ondevice training cpu packaging pipeline to a separated pipeline

  #21485 merged Jul 25, 2024

• Set CUDA12 as default in GPU packages

  #21438 merged Jul 25, 2024

• Update 05-performance.yml issue template to auto apply label

  #21486 merged Jul 25, 2024

• [VitisAI] use binary mode for context ep

  #21474 merged Jul 25, 2024

• OVEP - PR 1.19

  #21443 merged Jul 25, 2024

• Ignore ruff rule N813

  #21477 merged Jul 25, 2024

• Fix security issue #22016 #22017 #22018

  #21333 merged Jul 25, 2024

• Extend QDQPropagation transformer to handle multiple consumers

  #21313 merged Jul 24, 2024

• Update ruff and clang-format versions

  #21479 merged Jul 24, 2024

• [QNN EP] Update to QNN SDK 2.24.0

  #21463 merged Jul 24, 2024

• Update copy_strip_binary.sh: use "make install" instead

  #21464 merged Jul 24, 2024

• CoreML: Add ML Program ConvTranspose

  #21416 merged Jul 24, 2024

• [QNN EP] Improve QNN error reporting using the error message

  #21458 merged Jul 24, 2024

• [WebNN EP] ConvTranspose should calculate the pads or output shape

  #21292 merged Jul 24, 2024

• CoreML: Add GridSample ML Program support

  #21431 merged Jul 24, 2024

• [Fix] C++ API SetOutputShape for register custom op.

  #21366 merged Jul 23, 2024

• fix python qnn pipelines issues

  #21462 merged Jul 23, 2024

• [CUDA] Fix cuda provider fallback inconsistency

  #21425 merged Jul 23, 2024

• Update nodejs's cmake file to fix a file copy issue

  #21390 merged Jul 23, 2024

• Update C++ dependencies

  #21410 merged Jul 23, 2024

• CoreML: ML Program Slice

  #21433 merged Jul 23, 2024

• Update DirectML from 1.14.1 to 1.15.0

  #21323 merged Jul 22, 2024

• Adds ATen fallback for scaled_dot_product_attention

  #21107 merged Jul 22, 2024

• Fix typos according to reviewdog report.

  #21335 merged Jul 22, 2024

• Replace inline pip install with pip install from requirements*.txt

  #21106 merged Jul 22, 2024

• [WebNN EP] Add outputDataType option for the ArgMax/ArgMin ops

  #21385 merged Jul 22, 2024

• [CUDA] FusedMHARunnerFP16v2 thread-safe

  #21420 merged Jul 22, 2024

24 Pull requests opened by 20 people

• Fix typo: Complete the link symbol in pose-detection.md

  #21437 opened Jul 22, 2024

• Phi3 Android App tutorial

  #21446 opened Jul 22, 2024

• [WIP] Out-Tree EP feature

  #21450 opened Jul 23, 2024

• hack ext data location to reduce qd matmul memory usage

  #21451 opened Jul 23, 2024

• CoreML: Add ML Program Split Op

  #21456 opened Jul 23, 2024

• [js/webgpu] Add activation for conv3d naive

  #21466 opened Jul 23, 2024

• Update build from source instructions

  #21468 opened Jul 24, 2024

• [Fix] ShapeInferContext GetAttrxxxs support empty value

  #21471 opened Jul 24, 2024

• Update QNN pipeline pool

  #21482 opened Jul 24, 2024

• Create new_labeler.ylm

  #21488 opened Jul 24, 2024

• Add reduce kernels for bigger types

  #21490 opened Jul 25, 2024

• Propagate NaNs in the CPU min and max operators

  #21492 opened Jul 25, 2024

• Enable FP16 Clip and Handle Bias in FP16 Depthwise Conv

  #21493 opened Jul 25, 2024

• Create new stale issue workflow

  #21495 opened Jul 25, 2024

• [CUDA] Fix DecoderMaskedMultiHeadAttention bias input check

  #21498 opened Jul 25, 2024

• Add topology sort and remove useless cast nodes to fp16 conversion script

  #21499 opened Jul 25, 2024

• Bump torch from 1.13.1 to 2.2.0 in /tools/ci_build/github/windows/eager

  #21505 opened Jul 25, 2024

• [WebNN EP] Create MLGraphBuilder for every model builder

  #21514 opened Jul 26, 2024

• [WebNN EP] Add labels for all WebNN operators

  #21516 opened Jul 26, 2024

• Remove references of use-android-ndk.yml

  #21522 opened Jul 26, 2024

• Add Interactive Decoding support in GQA

  #21523 opened Jul 26, 2024

• [CUDA] Special case for K==0 in CUDA MatMul

  #21525 opened Jul 26, 2024

• Remove unused parameters from win-ci-vs-2022-job.yml

  #21530 opened Jul 27, 2024

• [js/web] allow load WebAssembly binary from buffer

  #21534 opened Jul 28, 2024

17 Issues closed by 8 people

• [Documentation] hopefully final logic app test - please ignore

  #21511 closed Jul 25, 2024

• [Documentation] [IGNORE] testing logic app change

  #21481 closed Jul 25, 2024

• [TEST ISSUE] Using this issue to test logic app - please ignore

  #21509 closed Jul 25, 2024

• [Feature Request] onnxruntime-node support for FreeBSD.

  #21508 closed Jul 25, 2024

• [Documentation] Community blog post contribution

  #21389 closed Jul 25, 2024

• [Build] long paths in NuGet package breaking build on Windows

  #21369 closed Jul 25, 2024

• Create Custom Node in CUDA

  #21442 closed Jul 25, 2024

• Custom Op Library does not work for CUDA

  #21417 closed Jul 25, 2024

• EP Error /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:123

  #21435 closed Jul 24, 2024

• [Documentation] Please Ignore - Using this issue to test broken logic app

  #21470 closed Jul 24, 2024

• [Documentation] Testing logic app update (please ignore)

  #21465 closed Jul 24, 2024

• TensorRT EP failed to create engine from network.

  #21415 closed Jul 24, 2024

• [Build] MSVC shared runtime linking error for TRT, CUDA, OpenVino and DML build

  #19642 closed Jul 23, 2024

• [TensorRT] Caching to a dedicated ONNX file does not work

  #21307 closed Jul 23, 2024

• [Build] Missing DirectML build in 1.18.1

  #21460 closed Jul 23, 2024

• Unable to append DML Provider

  #21432 closed Jul 22, 2024

• [Build] Cross compilation of the onnxruntime 1.5.1 for ARMv7 32bit target for gcc 4.9.2

  #21439 closed Jul 22, 2024

26 Issues opened by 25 people

• Pushing Rust bindings forward

  #21533 opened Jul 28, 2024

• [Build] Docerfile.cuda docker image build error

  #21532 opened Jul 28, 2024

• MLAS failing with "Could not find an implementation for QLinearMatMul"

  #21531 opened Jul 28, 2024

• CUDA_PATH is set but CUDA wasnt able to be loaded

  #21527 opened Jul 27, 2024

• Model saved by ORT as external data format will not be aligned for mapfile support

  #21524 opened Jul 26, 2024

• Error converting Microsoft Phi3 model to ONNX using Python and Transformers

  #21518 opened Jul 26, 2024

• [Build] detect nothing.i use opencv4.9 onnxruntime 1.16.1, it detect nothing

  #21513 opened Jul 26, 2024

• [Feature Request] Introduce get_available_initializers

  #21503 opened Jul 25, 2024

• Onnxruntime LoadLibrary failed with error 126

  #21501 opened Jul 25, 2024

• [Build] error cross compiling

  #21500 opened Jul 25, 2024

• [Performance] DequantizeLinear, pad and QuantizeLinear operation is not fused

  #21496 opened Jul 25, 2024

• Android build: Execution failed for task ':app:mergeExtDexDebug'.

  #21494 opened Jul 25, 2024

• failing to find trt_timing_cache_path

  #21484 opened Jul 24, 2024

• [CUDA, DML] MatMul does not properly handle matrices with inner dim == 0

  #21483 opened Jul 24, 2024

• [Performance] The 16-bit quantization QDQ model cannot be accelerated by CUDA

  #21478 opened Jul 24, 2024

• quant_pre_process failed on NonMaxSuppression

  #21476 opened Jul 24, 2024

• Dll version of Microsoft.ML.OnnxRuntime.dll is 0.0.0.0

  #21475 opened Jul 24, 2024

• [Web] Conv_token_460" failed. Error: Unsupported activation Tanh

  #21467 opened Jul 23, 2024

• Activate thread pool will cause crash.

  #21461 opened Jul 23, 2024

• TensorRT EP's inference results are abnormal.

  #21457 opened Jul 23, 2024

• Incorrect NaN handling for Min and Max operators on CPU with a single element input

  #21455 opened Jul 23, 2024

• [Web] Error: Tensor's size(512) does not match data length(1024)

  #21454 opened Jul 23, 2024

• [Feature Request] Memory Commit Savings. Possible total memory savings. Allow fully optimized model to be serialized to disk and used as-is without large heap allocs

  #21448 opened Jul 22, 2024

• [Build] ADD_LIBRARY cannot create target "memory" because another target with the same name already exists between xnnpack and absl

  #21441 opened Jul 22, 2024

• Quantization failed! The onnxruntime.quantization.quantize_dynamic seems didn't convert to the qint8 .onnx file successfully

  #21440 opened Jul 22, 2024

• FAIL : LoadLibrary failed with error 126 "" when trying to load "C:\code\Blueprint.Net.Server\bin\Debug\net8.0-windows10.0.22621.0\runtimes\win-x64\native\onnxruntime_providers_cuda.dll" ”

  #21436 opened Jul 22, 2024

67 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [WebNN EP] Enable IO Bindings with MLBuffer

  #21301 commented on Jul 24, 2024 • 27 new comments

• Update react-native to 0.74 and run npm audit fix

  #21122 commented on Jul 27, 2024 • 6 new comments

• Enable export for inference when eval model is loaded from buffer

  #21422 commented on Jul 25, 2024 • 5 new comments

• Keep QDQ nodes w/ nonpositive scale around MaxPool

  #21182 commented on Jul 27, 2024 • 5 new comments

• Drop QDQ around more nodes

  #21376 commented on Jul 26, 2024 • 4 new comments

• Create CMake option `onnxruntime_USE_VCPKG`

  #21348 commented on Jul 27, 2024 • 2 new comments

• Enable AVX NE CONVERT for FP16 to FP32 cast

  #21183 commented on Jul 28, 2024 • 2 new comments

• Adding CUDNN Frontend and use for CUDA NN Convolution

  #19470 commented on Jul 25, 2024 • 1 new comment

• Added WebNN Intro and Tutorial

  #20719 commented on Jul 24, 2024 • 1 new comment

• Refactor onnxruntime_fetchcontent_makeavailable cmake function

  #21328 commented on Jul 24, 2024 • 1 new comment

• [Feature Request] MPS provider

  #21271 commented on Jul 28, 2024 • 0 new comments

• [Performance] CoreML not being used to it's fullest capacity - custom transformer

  #19887 commented on Jul 28, 2024 • 0 new comments

• Cannot create arena allocator with Environment::CreateAndRegisterAllocator on MAC M2 with clang

  #21191 commented on Jul 28, 2024 • 0 new comments

• [Build] How to build for Android armeabi platform?

  #21192 commented on Jul 28, 2024 • 0 new comments

• Can onnxruntime.quantization.quantize_dynamic() work with onnx-trt?

  #21169 commented on Jul 28, 2024 • 0 new comments

• Could not load library cudnn_cnn_infer64_8.dll. Error code 127

  #18973 commented on Jul 28, 2024 • 0 new comments

• [Web] WebGPU and WASM Backends Unavailable within Service Worker

  #20876 commented on Jul 28, 2024 • 0 new comments

• Support Numpy v2.0

  #21063 commented on Jul 28, 2024 • 0 new comments

• [Feature Request] Request grid_sample 5D support 🌟

  #21382 commented on Jul 28, 2024 • 0 new comments

• [Jvm] Native crash during createSession: std::bad_cast

  #21147 commented on Jul 28, 2024 • 0 new comments

• Enabling c++20 on linux

  #17816 commented on Jul 24, 2024 • 0 new comments

• Not able to load onnx model multilingual-e5-large

  #21321 commented on Jul 24, 2024 • 0 new comments

• [js/node] enable float16 support for Node.js binding

  #20581 commented on Jul 28, 2024 • 0 new comments

• Mlas int4 int8 with avx2/512

  #20687 commented on Jul 26, 2024 • 0 new comments

• Implementation of Set Membership in TreeEnsemble

  #21222 commented on Jul 24, 2024 • 0 new comments

• [JS/WegGPU] Initial changes to support wasm64.

  #21260 commented on Jul 26, 2024 • 0 new comments

• [WebNN EP] Support ConvTranspose for TFLite backend

  #21291 commented on Jul 26, 2024 • 0 new comments

• [VitisAI] Remove shape infer from bridge ort

  #21331 commented on Jul 26, 2024 • 0 new comments

• add registered custom op for perf test

  #21336 commented on Jul 25, 2024 • 0 new comments

• Fix wrong per-tensor quantized weight type for matmul

  #21347 commented on Jul 27, 2024 • 0 new comments

• Remove tools/ci_build/github/android/run_nnapi_code_coverage.sh

  #21371 commented on Jul 27, 2024 • 0 new comments

• Add support tensor element type for register custom op shape infer function

  #21387 commented on Jul 24, 2024 • 0 new comments

• Upgrade emsdk from 3.1.59 to 3.1.62

  #21421 commented on Jul 24, 2024 • 0 new comments

• feat(onnxruntime-web): Allow the WASM backend to import the emscripten Module via a user-land defined loader

  #21430 commented on Jul 22, 2024 • 0 new comments

• [Build] Build python interface for Onnxruntime-qnn on aarch64 Linux

  #21203 commented on Jul 24, 2024 • 0 new comments

• New restricted asymmetric quantization mode in QDQ mode with zero_point restricted to either 128 or 0

  #21398 commented on Jul 24, 2024 • 0 new comments

• [Feature Request] Mark as negative tests for minimal CUDA build

  #21394 commented on Jul 24, 2024 • 0 new comments

• [Feature Request] 4bit and 2bit and 1bit quantization support

  #14997 commented on Jul 24, 2024 • 0 new comments

• [CUDA] Acquiring a CUDA allocator without loading a session.

  #19420 commented on Jul 23, 2024 • 0 new comments

• Microsoft.ML.OnnxRuntime.Gpu not working in MAUI project

  #14974 commented on Jul 23, 2024 • 0 new comments

• CUDA Graph Error - CUDA failure 900: operation not permitted when stream is capturing

  #15002 commented on Jul 23, 2024 • 0 new comments

• [Feature Request] Missing optimization of DequantizeLinear ∘ Flatten ∘ QuantizeLinear?

  #21375 commented on Jul 23, 2024 • 0 new comments

• Trilu op still not work with INT32 input

  #21400 commented on Jul 23, 2024 • 0 new comments

• [Documentation] The documentation for early versions is missing

  #20850 commented on Jul 23, 2024 • 0 new comments

• TensorrtExecutionProvider slower than CUDAExecutionProvider: Faster-rcnn [Performance]

  #17434 commented on Jul 23, 2024 • 0 new comments

• [Feature Request] Support for Florence-2 model family

  #21118 commented on Jul 23, 2024 • 0 new comments

• onnxruntime 在C++上如何实现fp16的推理 yolov5模型

  #20395 commented on Jul 23, 2024 • 0 new comments

• [Mobile] React-native OnnxruntimeJSIHelper install segfaults when registering functions

  #21003 commented on Jul 22, 2024 • 0 new comments

• [Web] `Error: [WebGPU] Kernel "[Conv] /text_encoder/encoder/layers.0/feed_forward/conv_2/Conv" failed. Error: FILTER_IN_CHANNEL should be equal to DATA_CHANNEL`

  #21108 commented on Jul 22, 2024 • 0 new comments

• DirectML Exception 80070057 "The parameter is incorrect"

  #20575 commented on Jul 21, 2024 • 0 new comments

• [Performance] Mapfile support for certain external data files is not working

  #21195 commented on Jul 27, 2024 • 0 new comments

• [Training] [ShapeInferenceError] Dimension could not be inferred: incompatible shapes

  #21327 commented on Jul 26, 2024 • 0 new comments

• [Performance] Whisper model inference results incorrect after Transformer Optimizer

  #21150 commented on Jul 26, 2024 • 0 new comments

• [Feature Request] C# Float16 DEMO, and float convert api

  #14303 commented on Jul 26, 2024 • 0 new comments

• How to do multithreaded infer with onnxruntime

  #21419 commented on Jul 25, 2024 • 0 new comments

• CUDA provider fallback to CPU is not working when CUDA_PATH environment variable exists

  #21424 commented on Jul 25, 2024 • 0 new comments

• using TensorRT EP by nuget

  #21428 commented on Jul 25, 2024 • 0 new comments

• [Web] Failed to compile shader on WebGL

  #12927 commented on Jul 25, 2024 • 0 new comments

• How to convert quantized ONNX model from Tensor-Oriented format to Operator-Oriented format?

  #21137 commented on Jul 25, 2024 • 0 new comments

• Quantized ONNX Model Still Has Float32 Input/Output Tensors

  #21138 commented on Jul 25, 2024 • 0 new comments

• [Training] Onnxruntime-training 1.18.0 for windows not available

  #21149 commented on Jul 25, 2024 • 0 new comments

• [E:onnxruntime:, qnn_execution_provider.cc:591 GetCapability] QNN SetupBackend failed qnn_backend_manager.cc:334 InitializeBackend Failed to initialize backend

  #21157 commented on Jul 25, 2024 • 0 new comments

• [Feature Request] SpaceToDepth & DepthToSpace integer implementations

  #21287 commented on Jul 25, 2024 • 0 new comments

• [Feature Request] Implement dynamically sized CPU sets for linux.

  #21241 commented on Jul 25, 2024 • 0 new comments

• Importing onnxruntime on AWS Lambdas with ARM64 processor causes crash

  #10038 commented on Jul 25, 2024 • 0 new comments

• [Performance] Multiple Sessions on Same GPU is very slow

  #21365 commented on Jul 25, 2024 • 0 new comments

• Inference using the CUDA EP returns nan

  #15752 commented on Jul 24, 2024 • 0 new comments