(Taking NNPACK for packaging attempt) |
|||
(13 intermediate revisions by 6 users not shown) | |||
Line 20: | Line 20: | ||
|- | |- | ||
| '''''In Progress''''' || Someone is currently working to package an appropriate version of the software | | '''''In Progress''''' || Someone is currently working to package an appropriate version of the software | ||
|- | |||
| Needs Review || Package is ready for review | |||
|} | |} | ||
= Dependency List = | = Dependency List = | ||
Line 28: | Line 30: | ||
|+ PyTorch Dependency List | |+ PyTorch Dependency List | ||
|- | |- | ||
! PyTorch Name !! Version !! Status !! Packager !! Fedora Package Name !! Fedora Package URL !! EPEL !! Notes | ! PyTorch Name !! Version !! Status !! Packager !! Fedora Package Name !! Fedora Package URL !! EPEL !! Notes !! Accelerator | ||
|- | |- | ||
| [https://github.com/Maratyszcza/ | | [https://github.com/Maratyszcza/FP16 FP16] || [https://github.com/Maratyszcza/FP16/tree/4dfe081cf6bcd15db339cf2680b9281b8451eeb3 4dfe081] (2020-05-14) || Packaged || trix || FP16 || [https://src.fedoraproject.org/rpms/FP16 FP16] || || || | ||
|- | |- | ||
| [https://github.com/Maratyszcza/ | | [https://github.com/Maratyszcza/FXdiv FXdiv] || [https://github.com/Maratyszcza/FXdiv/tree/b408327ac2a15ec3e43352421954f5b1967701d1 b408327] (2020-04-17) || Packaged || trix || fxdiv || [https://src.fedoraproject.org/rpms/fxdiv fxdiv] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/Maratyszcza/NNPACK NNPACK] || [https://github.com/Maratyszcza/NNPACK/tree/c07e3a0400713d546e0dea2d5466dd22ea389c73 c07e3a0] (2020-12-21) || '''''In Progress''''' || thunderbirdtr || || || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/pytorch/qnnpack QNNPACK] || [https://github.com/pytorch/QNNPACK/tree/7d2a4e9931a82adc3814275b6219a03e24e36b4c 7d2a4e9] (2019-08-28) || || || || || || Archived upstream on 2020-10-01 || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator VulkanMemoryAllocator] || [https://github.com/GPUOpen-LibrariesAndSDKs/VulkanMemoryAllocator/tree/a6bfc237255a6bac1513f7c1ebde6d8aed6b5191 v3.0.1] || Packaged || [[User:Mystro256|Jeremy Newton]] || VulkanMemoryAllocator || [https://src.fedoraproject.org/rpms/VulkanMemoryAllocator VulkanMemoryAllocator] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/google/benchmark benchmark] || [https://github.com/google/benchmark/tree/0d98dba29d66e93259db7daa53a9327df767a415 v1.6.1] || Packaged? || || google-benchmark || [https://src.fedoraproject.org/rpms/google-benchmark google-benchmark] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/pytorch/cpuinfo cpuinfo] || [https://github.com/pytorch/cpuinfo/tree/6481e8bef08f606ddd627e4d3be89f64d62e1b8a 6481e8b] (2023-01-13) || Packaged || trix || cpuinfo || [https://src.fedoraproject.org/rpms/cpuinfo cpuinfo] || || || | ||
|- | |- | ||
| [https://github.com/NVIDIA/ | | [https://github.com/NVIDIA/cub cub] || [https://github.com/NVlabs/cub/tree/d106ddb991a56c3df1b6d51b2409e36ba8181ce4 d106ddb] (2020-05-12) || || || || || || the linked repo is marked as moved, this could be a mess. [https://github.com/NVIDIA/cub/commit/d106ddb991a56c3df1b6d51b2409e36ba8181ce4 equivalent commit in NVIDIA/cub?] might be [https://github.com/NVIDIA/cub/commit/2a231db3226a9bfcd008bb6120bec12fe0a98cd1 cub 1.10.0] but the tags don't seem to correlate with versions mentioned in commit messages || | ||
|- | |- | ||
| [https://github.com/NVIDIA/ | | [https://github.com/NVIDIA/cudnn-frontend cudnn_frontend] || [https://github.com/NVIDIA/cudnn-frontend/tree/12f35fa2be5994c1106367cac2fba21457b064f4 v0.9.2] (2023-07-13) || || || || || || this requires CUDA and is likely not package-able for Fedora || | ||
|- | |- | ||
| [https:// | | [https://github.com/NVIDIA/cutlass cutlass] || [https://github.com/NVIDIA/cutlass/tree/6f47420213f757831fae65c686aa471749fa8d60 3.1] || || || || || || this requires CUDA and is likely not package-able for Fedora. we'll have to figure out how to build pytorch without it. || | ||
|- | |- | ||
| [https:// | | [https://eigen.tuxfamily.org/index.php?title=Main_Page eigen] || [https://gitlab.com/libeigen/eigen/-/tree/3147391d946bb4b6c68edd901f2add6ac1f31f8c 3.4.0] (2021-08-18) || Packaged? || || eigen3 || [https://src.fedoraproject.org/rpms/eigen3 eigen3] || || || | ||
|- | |- | ||
| [https:// | | [https://github.com/pytorch/FBGEMM fbgemm] || [https://github.com/pytorch/FBGEMM/tree/v0.5.0 0.5.0] || '''''In Progress''''' || aekoroglu || || || || || | ||
|- | |- | ||
| [https:// | | [https://flatbuffers.dev/ flatbuffers] || [https://github.com/google/flatbuffers/tree/01834de25e4bf3975a9a00e816292b1ad0fe184b 23.3.3] || Packaged? || || flatbuffers || [https://src.fedoraproject.org/rpms/flatbuffers flatbuffers] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/fmtlib/fmt fmt] || [https://github.com/fmtlib/fmt/tree/f5e54359df4c26b6230fc61d38aa294581393084 10.1.1] || Packaged? || || fmt || [https://src.fedoraproject.org/rpms/fmt fmt] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/houseroad/foxi foxi] || [https://github.com/houseroad/foxi/tree/c278588e34e535f0bb8f00df3880d26928038cad c278588] (2021-05-26) || || || || || || master/HEAD, appears to be an extension of [https://xadupre.github.io/draft/onnxcustom/onnxmd/onnx_docs/ONNXIFI.html ONNX Interface for Framework Integration] || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/google/gemmlowp gemmlowp] || [https://github.com/google/gemmlowp/tree/3fb5c176c17c765a3492cd2f0321b0dab712f350 3fb5c17] (2018-11-26) || "Needs review" || crisdel || gemmlowp || [https://copr.fedorainfracloud.org/coprs/crisdel/gemmlowp/build/6589799/ gemmlowp] || || Needs review || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/facebookincubator/gloo gloo] || [https://github.com/facebookincubator/gloo/tree/cf1e1abc95d0b961222ee82b6935f76250fbcf16 cf1e1ab] (2023-07-19) || Packaged || trix || || || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/google/googletest googletest] || [https://github.com/google/googletest/commits/e2239ee6043f73722e7aa812a459f54a28552929 v1.11.0] (2021-06-11) || Packaged? || || gtest || [https://src.fedoraproject.org/rpms/gtest gtest] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/intel/ideep ideep] || [https://github.com/intel/ideep/commits/13dd1fdc6217c9eccfa7166798dd81a0476e63dd 13dd1fd] || || || || || || master branch appears dead - there appear to be two current branches: ideep_dev and ideep_pytorch and the 13dd1fd commit appearst to only be in the ideep_pytorch branch || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/Yangqing/ios-cmake ios-cmake] || [https://github.com/Yangqing/ios-cmake/tree/8abaed637d56f1337d6e1d2c4026e25c1eade724 8abaed6] (2017-11-15)|| || || || || || fork of [https://github.com/ollef/ios-cmake ollef/ios-cmake] which is a fork of [https://github.com/cristeab/ios-cmake cristeab/ios-cmake] || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/intel/ittapi ittapi] || [https://github.com/intel/ittapi/tree/5b8a7d7422611c3a0d799fb5fc5dd4abfae35b42 5b8a7d7] (2022-04-12) || Submitted to review || aekoroglu || || || || [https://bugzilla.redhat.com/show_bug.cgi?id=2248952 RHBZ #2248952] || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/pytorch/kineto kineto] || [https://github.com/pytorch/kineto/tree/49e854d805d916b2031e337763928d2f8d2e1fbf 49e854d] (2023-08-08) || || || || || || well after last release (v0.4.0) || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/microsoft/mimalloc mimalloc] || [https://github.com/microsoft/mimalloc/tree/b66e3214d8a104669c2ec05ae91ebc26a8f5ab78 b66e321] (2023-04-24) || Packaged? || || mimalloc || [https://src.fedoraproject.org/rpms/mimalloc mimalloc] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/richgel999/miniz miniz-2.1.0] || [https://github.com/pytorch/pytorch/tree/main/third_party/miniz-2.1.0 Forked in PyTorch Upstream] || Packaged? || || miniz || [https://src.fedoraproject.org/rpms/miniz miniz] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/NVIDIA/nccl nccl] || [https://github.com/NVIDIA/nccl/tree/559b70f86c190a0d8f67f0d7a0f2c9810dd1e8c7 2.18.5-1] (2023-08-23) || || || || || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/intel/ARM_NEON_2_x86_SSE neon2sse] || [https://github.com/intel/ARM_NEON_2_x86_SSE/tree/97a126f08ce318023be604d03f88bf0820a9464a 97a126f] (2018-09-17) || Submitted to review || aekoroglu || || || || [https://bugzilla.redhat.com/show_bug.cgi?id=2248911 RHBZ #2248911] || | ||
|- | |- | ||
| [https:// | | [https://github.com/nlohmann/json nlohmann/json] || [https://github.com/nlohmann/json/tree/87cda1d6646592ac5866dc703c8e1839046a6806 87cda1d] (2022-06-18) || Packaged? || || json || [https://src.fedoraproject.org/rpms/json json] || || between 3.10.5 and 3.11.0 || | ||
|- | |- | ||
| [https:// | | [https://pytorch.org/blog/introducing-nvfuser-a-deep-learning-compiler-for-pytorch/ nvfuser] || [https://github.com/pytorch/pytorch/tree/main/third_party/nvfuser in pytorch tree] || || || || || || pytorch tree has no nvfuser ??? || | ||
|- | |- | ||
| [https://github.com/onnx/onnx | | [https://github.com/onnx/onnx onnx] || [https://github.com/onnx/onnx/commits/1014f41f17ecc778d63e760a994579d96ba471ff 1.14.1] || Packaged? || || onnx || [https://src.fedoraproject.org/rpms/onnx onnx] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/onnx/onnx-tensorrt/ onnx-tensorrt] || [https://github.com/onnx/onnx-tensorrt/commits/c153211418a7c57ce071d9ce2a41f8d1c85a878f 6.0] (2019-09-16) || || || || || || this requires CUDA and is likely not package-able for Fedora || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/mreineck/pocketfft pocketfft] || [https://github.com/mreineck/pocketfft/tree/ea778e37710c07723435b1be58235996d1d43a5a release_for_eigen] (2021-03-12) || Packaged || trix || pocketfft ||[https://src.fedoraproject.org/rpms/pocketfft pocketfft] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/protocolbuffers/protobuf protobuf] || [https://github.com/protocolbuffers/protobuf/tree/d1eca4e4b421cd2997495c4b4e65cea6be4e9b8a d1eca4e] (2020-10-08)|| Packaged? || || protobuf || [https://src.fedoraproject.org/rpms/protobuf protobuf] || || after v3.13.0, before v3.14.0-rc1 || | ||
|- | |- | ||
| [https://github.com/Maratyszcza/ | | [https://github.com/Maratyszcza/psimd psimd] || [https://github.com/Maratyszcza/psimd/tree/072586a71b55b7f8c584153d223e95687148a900 current master/HEAD] (2020-05-17) || Packaged || trix || psimd || [https://src.fedoraproject.org/rpms/psimd psimd] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/Maratyszcza/pthreadpool pthreadpool] || [https://github.com/Maratyszcza/pthreadpool/tree/a134dd5d4cee80cce15db81a72e7f929d71dd413 a134dd5] (2021-04-13) || Packaged || trix || || || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/pybind/pybind11 pybind11] || [https://github.com/pybind/pybind11/tree/8a099e44b3d5f85b20f05828d919d2332a8de841 2.11.1] || Packaged? || || pybind11 || [https://src.fedoraproject.org/rpms/pybind11 pybind11] || || || | ||
|- | |- | ||
| | | [https://github.com/malfet/PeachPy python-peachpy] || [https://github.com/malfet/PeachPy/tree/f45429b087dd7d5bc78bb40dc7cf06425c252d67 f45429b] || || || || || || This is a fork of [https://github.com/Maratyszcza/PeachPy Maratyszcza/PeachPy] || | ||
|- | |- | ||
| | | six || [https://github.com/pytorch/pytorch/blob/main/BUILD.bazel#L1767 no specified version] || Packaged? || || python-six || [https://src.fedoraproject.org/rpms/python-six python-six] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/shibatch/sleef sleef] || [https://github.com/shibatch/sleef/tree/e0a003ee838b75d11763aa9c3ef17bf71a725bff e0a003e] (2020-12-21) || Packaged? || || sleef || [https://src.fedoraproject.org/rpms/sleef sleef] || || || | ||
|- | |- | ||
| [https://github.com/ | | [https://github.com/oneapi-src/oneTBB tbb] || [https://github.com/oneapi-src/oneTBB/tree/a51a90bc609bb73db8ea13841b5cf7aa4344d4a9 TBB 2018 U6?] (2018-10-09)|| Packaged? || || tbb || [https://src.fedoraproject.org/rpms/tbb tbb] || || || | ||
|- | |- | ||
| [https://github.com/pytorch/ | | [https://github.com/pytorch/pytorch/tree/main/third_party/tensorflow_cuda_bazel_build/cuda tensorflow_cuda_bazel_build] || || || || || || || might be part of the pytorch build process || | ||
|- | |- | ||
| | | [https://github.com/pytorch/tensorpipe/ tensorpipe] || [https://github.com/pytorch/tensorpipe/tree/52791a2fd214b2a9dc5759d36725909c1daa7f2e 52791a2] || || || || || || Archived upstream on 2023-07-01 || | ||
|- | |- | ||
| [https://github.com/google/XNNPACK XNNPACK] || [https://github.com/google/XNNPACK/tree/51a987591a6fc9f0fc0707077f53d763ac132cbf 51a9875] || | | [https://github.com/google/XNNPACK XNNPACK] || [https://github.com/google/XNNPACK/tree/51a987591a6fc9f0fc0707077f53d763ac132cbf 51a9875] || Packaged || trix || || || || || | ||
|- | |- | ||
| [https://github.com/facebook/zstd zstd] || [https://github.com/facebook/zstd/tree/aec56a52fbab207fc639a1937d1e708a282edca8 v1.3.2 + 1 month] || Packaged? || || zstd || [https://src.fedoraproject.org/rpms/zstd zstd] || || | | [https://github.com/facebook/zstd zstd] || [https://github.com/facebook/zstd/tree/aec56a52fbab207fc639a1937d1e708a282edca8 v1.3.2 + 1 month] || Packaged? || || zstd || [https://src.fedoraproject.org/rpms/zstd zstd] || || || | ||
|} | |} |
Latest revision as of 13:37, 10 November 2023
PyTorch Dependency Packaging
This page was created for the purpose of tracking the effort for packaging PyTorch in Fedora.
If you start working on a package in this list, please mark it as In Progress and put your name under the packager column. Also, have your review request block the ML-SIG Review Tracker and send a link to the review request in at least the AI-ML matrix room.
If you find errors in this list (package that aren't actually needed or missing dependencies), feel free to add them to this list.
Status Key
When updating the dependency table, please use one of the values in the table below. If the status is really not encompassed by the existing statuses, please add a new value and description for the current state.
Value | Meaning |
---|---|
Packaged? | Software is packaged in Fedora but it may not be a version that is compatible with PyTorch |
Packaged | Software is packaged in Fedora and is compatible with building PyTorch |
In Progress | Someone is currently working to package an appropriate version of the software |
Needs Review | Package is ready for review |
Dependency List
The upstream links and versions were extracted from pytorch's third_party directory upstream on 2023-09-14.
PyTorch Name | Version | Status | Packager | Fedora Package Name | Fedora Package URL | EPEL | Notes | Accelerator |
---|---|---|---|---|---|---|---|---|
FP16 | 4dfe081 (2020-05-14) | Packaged | trix | FP16 | FP16 | |||
FXdiv | b408327 (2020-04-17) | Packaged | trix | fxdiv | fxdiv | |||
NNPACK | c07e3a0 (2020-12-21) | In Progress | thunderbirdtr | |||||
QNNPACK | 7d2a4e9 (2019-08-28) | Archived upstream on 2020-10-01 | ||||||
VulkanMemoryAllocator | v3.0.1 | Packaged | Jeremy Newton | VulkanMemoryAllocator | VulkanMemoryAllocator | |||
benchmark | v1.6.1 | Packaged? | google-benchmark | google-benchmark | ||||
cpuinfo | 6481e8b (2023-01-13) | Packaged | trix | cpuinfo | cpuinfo | |||
cub | d106ddb (2020-05-12) | the linked repo is marked as moved, this could be a mess. equivalent commit in NVIDIA/cub? might be cub 1.10.0 but the tags don't seem to correlate with versions mentioned in commit messages | ||||||
cudnn_frontend | v0.9.2 (2023-07-13) | this requires CUDA and is likely not package-able for Fedora | ||||||
cutlass | 3.1 | this requires CUDA and is likely not package-able for Fedora. we'll have to figure out how to build pytorch without it. | ||||||
eigen | 3.4.0 (2021-08-18) | Packaged? | eigen3 | eigen3 | ||||
fbgemm | 0.5.0 | In Progress | aekoroglu | |||||
flatbuffers | 23.3.3 | Packaged? | flatbuffers | flatbuffers | ||||
fmt | 10.1.1 | Packaged? | fmt | fmt | ||||
foxi | c278588 (2021-05-26) | master/HEAD, appears to be an extension of ONNX Interface for Framework Integration | ||||||
gemmlowp | 3fb5c17 (2018-11-26) | "Needs review" | crisdel | gemmlowp | gemmlowp | Needs review | ||
gloo | cf1e1ab (2023-07-19) | Packaged | trix | |||||
googletest | v1.11.0 (2021-06-11) | Packaged? | gtest | gtest | ||||
ideep | 13dd1fd | master branch appears dead - there appear to be two current branches: ideep_dev and ideep_pytorch and the 13dd1fd commit appearst to only be in the ideep_pytorch branch | ||||||
ios-cmake | 8abaed6 (2017-11-15) | fork of ollef/ios-cmake which is a fork of cristeab/ios-cmake | ||||||
ittapi | 5b8a7d7 (2022-04-12) | Submitted to review | aekoroglu | RHBZ #2248952 | ||||
kineto | 49e854d (2023-08-08) | well after last release (v0.4.0) | ||||||
mimalloc | b66e321 (2023-04-24) | Packaged? | mimalloc | mimalloc | ||||
miniz-2.1.0 | Forked in PyTorch Upstream | Packaged? | miniz | miniz | ||||
nccl | 2.18.5-1 (2023-08-23) | |||||||
neon2sse | 97a126f (2018-09-17) | Submitted to review | aekoroglu | RHBZ #2248911 | ||||
nlohmann/json | 87cda1d (2022-06-18) | Packaged? | json | json | between 3.10.5 and 3.11.0 | |||
nvfuser | in pytorch tree | pytorch tree has no nvfuser ??? | ||||||
onnx | 1.14.1 | Packaged? | onnx | onnx | ||||
onnx-tensorrt | 6.0 (2019-09-16) | this requires CUDA and is likely not package-able for Fedora | ||||||
pocketfft | release_for_eigen (2021-03-12) | Packaged | trix | pocketfft | pocketfft | |||
protobuf | d1eca4e (2020-10-08) | Packaged? | protobuf | protobuf | after v3.13.0, before v3.14.0-rc1 | |||
psimd | current master/HEAD (2020-05-17) | Packaged | trix | psimd | psimd | |||
pthreadpool | a134dd5 (2021-04-13) | Packaged | trix | |||||
pybind11 | 2.11.1 | Packaged? | pybind11 | pybind11 | ||||
python-peachpy | f45429b | This is a fork of Maratyszcza/PeachPy | ||||||
six | no specified version | Packaged? | python-six | python-six | ||||
sleef | e0a003e (2020-12-21) | Packaged? | sleef | sleef | ||||
tbb | TBB 2018 U6? (2018-10-09) | Packaged? | tbb | tbb | ||||
tensorflow_cuda_bazel_build | might be part of the pytorch build process | |||||||
tensorpipe | 52791a2 | Archived upstream on 2023-07-01 | ||||||
XNNPACK | 51a9875 | Packaged | trix | |||||
zstd | v1.3.2 + 1 month | Packaged? | zstd | zstd |