Sysadmin/Project Metadata API: Difference between revisions

From KDE Community Wiki
m (→‎The Single Project Endpoint: typo fix: browser)
(Replaced content with "To conveniently query project metadata you can use the project API. * Base URI: https://projects.kde.org/api/ * [https://projects.kde.org/api/ API Documentation] * Bug/Fe...")
 
Line 1: Line 1:
{{ Remember | 2 = Important | 1 = The service described in this wiki page is not yet live. Information described in this page may not be taken as fact and is subject to change }}
To conveniently query project metadata you can use the project API.


= Introduction =
* Base URI: https://projects.kde.org/api/
* [https://projects.kde.org/api/ API Documentation]
* Bug/Feature reports are managed through sysadmin tickets https://go.kde.org/u/systickets


KDE has relied on a file called <code>kde_projects.xml</code> to provide metadata about all of KDE's various projects to people and applications that need it. For example, <code>kdesrc-build</code> uses it to know the location of the project in KDE's project hierarchy, as well as where it can find the source code to download and build. KDE's [https://build.kde.org/ Continuous Integration System] uses it to obtain much the same information, which it uses to automatically download, build and run tests on the software. The i18n team uses this file to map which translations go into which versions of the software.
{{ Remember | 2 = Important | 1 = The data structures are subject to expansion. A client implementation must not break if previously unknown JSON properties appear!  }}
 
However, <code>kde_projects.xml</code> is a mammoth '''1.4 MB''' text file that must be downloaded in its entirety and parsed by all software that need to use it, even if it needs information about just one project. The file contains a lot of extraneous information, and takes a long time to generate, due to which we cannot generate this file live as and when requested.
 
Therefore, we're deprecating the <code>kde_projects.xml</code> file and replacing it with a REST API, which should be simpler to use, faster to generate and provide the power to the requester to only request the data they need, nothing more and nothing less.
 
'''NOTE:''' The XML file is deprecated, but will continue to be updated every half-hour until at least '''31st December 2016, 23:59:59 UTC'''. After this deadline, the file will not be updated and may be removed at any time. This deadline may be extended based on the community's requirements.
 
= API Endpoints =
 
<pre>Endpoint Base: https://apps.kde.org/api/
Content Type: application/json</pre>
All endpoints in this API documentation is relative to <code>https://apps.kde.org/api/</code>. All requests and responses to and from this API are JSON encoded.
 
The API follows REST standards and returns standard HTTP status codes, as well as a JSON representation of the error if one occurs.
 
The API is neither rate-limited nor does it require authentication. None of the API's endpoints modify any data on our servers; therefore this API cannot be considered to be writable.
 
== The Components Endpoint ==
 
<pre>Endpoint: /components/
Methods: GET
Possible Return Codes: 200 OK
 
Sample Return:
[
    &quot;kdesupport&quot;,
    &quot;frameworks&quot;,
    &quot;extragear&quot;,
    &quot;kde&quot;,
    &quot;calligra&quot;,
    ...
]</pre>
This endpoint simply returns a list of all of KDE's major components. Use the Index API to explore the contents of each component
 
== The Index API ==
 
<pre>Endpoint: /index/[:path]
Methods: GET
Possible Return Codes: 200 OK, 404 Not Found
 
Sample Return (/index/kde/kdegraphics):
{
    &quot;gwenview&quot;: &quot;kde/kdegraphics/gwenview&quot;,
    &quot;okular&quot;: &quot;kde/kdegraphics/okular&quot;,
    &quot;kolourpaint&quot;: &quot;kde/kdegraphics/kolourpaint&quot;,
    &quot;spectacle&quot;: &quot;kde/kdegraphics/spectacle&quot;,
    &quot;libs&quot;: {
        &quot;libkexiv2&quot;: &quot;kde/kdegraphics/libs/libkexiv2&quot;,
        &quot;libkdcraw&quot;: &quot;kde/kdegraphics/libs/libkdcraw&quot;,
        ...
    },
    ...
}</pre>
This endpoint returns the hierarchy of projects rooted at the <code>path</code> supplied. The <code>path</code> is optional and may be omitted. For example, a request to <code>https://apps.kde.org/api/index/</code> (without a <code>path</code>) returns the entire project tree. A request to <code>https://apps.kde.org/api/index/frameworks</code> returns all projects under the <code>frameworks</code> component.
 
The return is a tree represented in a JSON Object. All non-leaf nodes are themselves subtrees, while all leaf nodes are valid JSON string values giving the complete path to that particular project. This path may be used as the argument to the Projects API.
 
== The Projects API ==
 
The Projects API can be used to obtain detailed data about one or more projects. This API has two endpoints which may either be used to obtain data about a single project only, or multiple projects at once.
 
=== The Single Project Endpoint ===
 
<pre>Endpoint: /project/[:path]/:project
Methods: GET
Possible Return Codes: 200 OK, 404 Not Found
 
Sample Return (/index/kde/kdegraphics/spectacle):
{
    &quot;name&quot;: &quot;Screenshot Capture Utility&quot;,
    &quot;description&quot;: &quot;The new screenshot capture utility, replaces KSnapshot&quot;,
    &quot;maintainers&quot;: [ { &quot;username&quot;: &quot;bgupta&quot;, &quot;displayname&quot;: &quot;Boudhayan Gupta&quot; }],
    &quot;weblinks&quot;: {
        &quot;gitweb&quot;: &quot;https://quickgit.kde.org/?p=spectacle.git&quot;,
        &quot;appsdb&quot;: &quot;https://apps.kde.org/projects/kde/kdegraphics/spectacle&quot;
    },
    &quot;repository&quot;: {
        &quot;hasRepo&quot;: true,
        &quot;isActive&quot;: true,
        &quot;repoType&quot;: &quot;git&quot;,
        &quot;repoUrls&quot;: {
            &quot;git&quot;: { &quot;access&quot;: &quot;read-only&quot;, &quot;url&quot;: &quot;git://anongit.kde.org/spectacle&quot; },
            &quot;ssh&quot;: { &quot;access&quot;: &quot;read+write&quot;, &quot;url&quot;: &quot;[email protected]:spectacle&quot; },
            ...
        },
        &quot;i18nBranches&quot;: {
            &quot;trunk&quot;: &quot;none&quot;,
            &quot;stable&quot;: &quot;none&quot;,
            &quot;trunk_kf5&quot;: &quot;master&quot;,
            &quot;stable_kf5&quot;: &quot;Applications/15.12&quot;
        },
        &quot;allBranches&quot;: [&quot;master&quot;, &quot;Applications/15.12&quot;, &quot;Applications/16.04&quot;]
    }
}</pre>
This endpoint provides detailed data about a single project. It contains basic data about the project, such as its name, description and a list of maintainers, as well as links to relevant human-readable HTML sites within the KDE infrastructure (such as a web-based Git repository browser, or a landing page for the project).
 
If the project has an associated repository, then the <code>repository.hasRepo</code> key will be <code>true</code>. This object will then carry additional information, such as links to the repository by protocol, a mapping between version control branches and i18n branches, and a list of all version control branches in the repository.
 
The above sample return contains all the data that can possibly be returned; all keys are listed.
 
=== The Multiple Projects Endpoint ===
 
<pre>Endpoint: /projects/
Methods: POST
Possible Return Codes: 200 OK, 404 Not Found
Payload: JSON array of paths to projects
 
Sample Request Payload:
[
    &quot;kde/kdegraphics/spectacle&quot;,
    &quot;frameworks/baloo&quot;,
    ...
]
 
Sample Return:
{
    &quot;kde/kdegraphics/spectacle&quot;: {
        &quot;name&quot;: &quot;Screenshot Capture Utility&quot;,
        &quot;description&quot;: &quot;The new screenshot capture utility, replaces KSnapshot&quot;,
        ...
    },
    &quot;frameworks/baloo&quot;: {
        &quot;name&quot;: &quot;Baloo&quot;,
        &quot;description&quot;: &quot;Baloo is a framework for searching and managing metadata.&quot;,
    },
    ...
}</pre>
Sometimes, it may be desirable to request metadata for multiple projects at once. Instead of making multiple requests for individual projects, it is possible to use this endpoint to send a POST request containing a JSON-encoded list of paths for which to request metadata. An object is returned, in which the key is the path to the project, and the value is the exact same object that is returned at the single-project endpoint.
 
Even if a single path in the array supplied in the payload is invalid, a <code>404 Not Found</code> error will be generated. The JSON representation of the error return will contain information about which path was invalid.
 
'''IMPORTANT:''' This request may take a not insignificant time to complete; the execution time rises linearly with the number of paths requested. Please make requests to this endpoint sparingly, and try to make it in small batches of paths. The application backing this service is not multithreaded and will block while serving a single request. While we do have a pool of multiple instances of this application running, long-running requests ''will'' result in one instance not being able to serve other requests while one completes. A request to generate data for all 2500+ projects in the system may take well over 30 seconds to complete.

Latest revision as of 09:09, 30 May 2017

To conveniently query project metadata you can use the project API.

 
Important
The data structures are subject to expansion. A client implementation must not break if previously unknown JSON properties appear!