RFC: tracking array API compliance
まだ誰も着手していません。
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 25/100
- issue の種類
- 機能追加
- 明瞭さ
- 説明が足りない
- 活発さ
- 停滞
- 技術スタック
- python
調査の方向性
RFC の Overview と Proposal セクションから始め、互換性 JSON の例と、参照されている array-library の追跡アプローチを確認してください。issue では実装ファイルもテストも指定されていません。完了とするには、合意されたプロセスと、対象となる API とライブラリをカバーする公開された互換性データが必要です。
索引モデルが issue の本文から書いたものです。
説明
This RFC seeks to propose a means for tracking array API compliance.
Overview
Currently, consumers of array libraries lack a centralized mechanism for determining whether any given array API is compliant with the array API specification.
Array libraries have implemented various means for tracking implementation progress:
-
MXNet
-
PyTorch
- spreadsheets
- tracking issues
- documents
- labels
-
NumPy
-
CuPy
-
Dask
-
(aggregated)
However, surfacing this information to understand how broadly an API is supported and in what version any given API was implemented requires knowing where to look, a significant investment of time and energy, and dogged investigation.
A significant barrier to specification adoption among downstream libraries is not knowing (a) what libraries currently implement any given API and (b) which array library versions are needed in order to access specification-compliant APIs.
This RFC seeks to address this barrier by providing a process for tracking array API specification compliance and making this information publicly available.
Proposal
This RFC proposes an approach similar to that of Web APIs whereby compatibility information is stored in JSON files and made publicly available on the web.
An example JSON file for the `asarray` API.
{
"asarray": {
"__compat__": {
"spec_url": "https://data-apis.org/array-api/latest/API_specification/generated/signatures.creation_functions.asarray.html#signatures.creation_functions.asarray",
"support": {
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy` kwarg is only partially implemented."
}
],
"dask": null,
"jax": null,
"mxnet": null,
"numpy": [
{
"version_added": "1.22.0",
"status": {
"experimental": true,
"deprecated": false,
"partial_implementation": true
},
"notes": "Provisionally available via `numpy.array_api`."
}
],
"pytorch": [
{
"version_added": "1.11.0",
"status":{
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"tensorflow": null
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
},
"dtype": {
"__compat__": {
"support": {
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"dask": null,
"jax": null,
"mxnet": null,
"numpy": [
{
"version_added": "1.22.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"pytorch": [
{
"version_added": "1.11.0",
"status":{
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"tensorflow": null
}
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
},
"device": {
"__compat__": {
"support": {
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"dask": null,
"jax": null,
"mxnet": null,
"numpy": [
{
"version_added": "1.22.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"pytorch": [
{
"version_added": "1.11.0",
"status":{
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"tensorflow": null
}
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
},
"copy": {
"__compat__": {
"support": {
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy=False` is not implemented"
}
],
"dask": null,
"jax": null,
"mxnet": null,
"numpy": [
{
"version_added": "1.22.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy=False` is not implemented"
}
],
"pytorch": [
{
"version_added": "1.11.0",
"status":{
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
}
],
"tensorflow": null
}
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
}
}
}
At a high level, for each API in the array API specification, there would be a corresponding JSON file containing compatibility data for each array library of interest.
{
"<api>": {
"__compat__": {
...,
"support": {
"cupy": [...],
"dask": null,
"jax": null,
"mxnet": null,
"numpy": [...],
"pytorch": [...],
"tensorflow": null,
...
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
},
...
}
}
The status field indicates whether an API is either on a standards track, is experimental and thus subject to change, or deprecated.
The status field is an object as the contained fields are not mutually exclusive (e.g., an experimental API could be deprecated after failing to gain sufficient traction during the specification process, or a standards track API could be deprecated due to obsolescence and replacement by a new API).
The support field maps array libraries to an implementation status. If an array library lacks even partial support, its corresponding field value is null.
For array libraries with partial or full support, the corresponding field value would be an array of objects having the following fields:
-
version_added: the version in which a specification-compliant API was added for a respective array library.
-
version_removed: the version in which a specification-compliant API was removed for a respective array library.
-
status: a status object having the following fields:
- experimental:
booleanindicating whether an API is exposed under an experimental/preview status and, thus, subject to possible change. - deprecated:
booleanindicating whether an array library has deprecated an API. - partial_implementation:
booleanindicating whether an API has only been partially implemented by an array library (e.g., partial kwarg support).
- experimental:
-
notes: a string (possibly containing Markdown) for providing additional information concerning implementation status.
The version_added and version_removed fields are mutually exclusive.
As an example, consider the following compliance data for CuPy and asarray.
...,
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy` kwarg is only partially implemented."
}
],
...
The above indicates that the asarray API was implemented in CuPy starting in version 10.0.0, is not exposed on an experimental status, and is only partially implemented. The notes clarify that the partial implementation status is due to the copy kwarg having incomplete support.
Suppose CuPy subsequently adds complete support for the copy kwarg in a subsequent version. In which case, the compliance data would be updated as follows:
...,
"cupy": [
{
"version_added": "10.1.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": false
},
"notes": ""
},
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy` kwarg is only partially implemented."
}
],
...
Notice that the partial_implementation flag and the clarifying notes have been removed. By storing the data in an array, we are able to track implementation progress over time.
In addition to total API compliance, this RFC proposes to break out support for each optional argument. Using asarray as an example,
{
"asarray": {
"__compat__": {...},
...,
"copy": {
"__compat__": {
"support": {
"cupy": [
{
"version_added": "10.0.0",
"status": {
"experimental": false,
"deprecated": false,
"partial_implementation": true
},
"notes": "`copy=False` is not implemented"
}
],
...
}
},
"status": {
"standard_track": true,
"experimental": false,
"deprecated": false
}
}
}
}
Compliance for optional arguments follows a similar structure as total API compliance. Namely, a special __compat__ field containing compliance data and a status field indicating the status of the API at the standards level.
Updating Compliance Data
Array library maintainers are best positioned to know both (a) when an API is implemented and (b) to what extent an API is compliant. Accordingly, array libraries should plan to dedicate a small amount of time updating compliance status for each release.
In the future, we can investigate automating this process. For example, array libraries could include compliance data in their release notes in a machine readable format which we can then use to generate automatic updates.
However, in the absence of such automation, this RFC proposes to rely on maintainers and crowdsourcing for ensuring that compliance data is up-to-date.
This RFC proposes that compliance data be stored in a standalone public Git repository against which contributors (including those outside of the Consortium) may open pull requests fixing or updating compliance entries.
Public Consumption
This RFC proposes to surface compliance data in a human-friendly manner by publishing this data directly in the publicly hosted specification.
The specification for each API should contain a table similar to the following:
In this example table, an individual is able to immediately infer how widely an API is implemented and to what extent implementations are specification-compliant.
For example, we can see that the asarray API is available in NumPy under an experimental status and has only partial support for the copy kwarg starting in version 1.22.0. CuPy has similar compliance; however, the API is not exposed experimentally. PyTorch has full compliance starting in version 1.11.0. All other libraries currently do not have stable releases exposing a specification-compliant asarray.
Questions
- Should we be collecting any additional data (e.g., device support)?
- Are we okay with the proposed data format and process?
- Will array libraries commit to helping keep compliance data up-to-date?
- 主要言語
- Python
- スター
- 281
- フォーク
- 52
- PR マージ指標
- 30日以内にマージされた PR はありません
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
data-apis/array-api のほかの issue
-
bug Maintenance Narrative Content
難易度 1/5 1〜3時間 初心者へのやさしさ 88/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
-
難易度 5/5 1週間以上 初心者へのやさしさ 35/100
-
Maintenance
-
難易度 5/5 1週間以上 初心者へのやさしさ 25/100
data-apis/array-api の issue をすべて見る
似ている issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
stephrobert/dsoxlab#238 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
sublimehq/package_control#1780 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
nwg-piotr/nwg-displays#145 ·