(→Scope: add font metadata cleanup) |
(And remove announced category) |
||
| (15 intermediate revisions by 3 users not shown) | |||
| Line 13: | Line 13: | ||
== Current status == | == Current status == | ||
* Targeted release: [[Releases/ | * Targeted release: [[Releases/21 | Fedora 21 ]] | ||
* Last updated: | * Last updated: 2014-01-18 | ||
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page | <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page | ||
Bugzilla states meaning as usual: | Bugzilla states meaning as usual: | ||
| Line 23: | Line 23: | ||
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | ||
--> | --> | ||
* Tracker bug: | * Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=F20WebAssets F20WebAssets] [https://bugzilla.redhat.com/show_bug.cgi?id=998583 #998583] | ||
== Detailed Description == | == Detailed Description == | ||
A standard directory (<code>/usr/share/web-assets</code>) for static bits that are intended to be delivered to web browsers, such as CSS Frameworks, UI libraries, etc. will be introduced. HTTP daemons in the distribution should make this directory available publicly as <code>/ | A standard directory (<code>/usr/share/web-assets</code>) for static bits that are intended to be delivered to web browsers, such as CSS Frameworks, UI libraries, etc. will be introduced. HTTP daemons in the distribution should make this directory available publicly as <code>/.sysassets</code>. | ||
Additionally, a standard directory (<code>/usr/share/javascript</code>) for JavaScript code, which may be used in browsers or server-side by applications such as nodejs or rubygem-execjs will also be introduced. | Additionally, a standard directory (<code>/usr/share/javascript</code>) for JavaScript code, which may be used in browsers or server-side by applications such as nodejs or rubygem-execjs will also be introduced. | ||
| Line 40: | Line 40: | ||
== Scope == | == Scope == | ||
=== The <code>web-assets</code> RPM === | === Fedora 20 === | ||
This section describes tasks already completed during the Fedora 20 release cycle. | |||
==== The <code>web-assets</code> RPM ==== | |||
'''Status:''' [https://bugzilla.redhat.com/show_bug.cgi?id=997678 built] | |||
A <code>web-assets</code> package will be introduced. [https://github.com/tchollingsworth/fedora-web-assets A preliminary SRPM is available here.] | A <code>web-assets</code> package will be introduced. [https://github.com/tchollingsworth/fedora-web-assets A preliminary SRPM is available here.] | ||
| Line 56: | Line 62: | ||
A <code>web-assets-devel</code> subpackage will be provided that provides macros like <code>%{_assetdir}</code> and other conveniences described in the proposed guidelines. | A <code>web-assets-devel</code> subpackage will be provided that provides macros like <code>%{_assetdir}</code> and other conveniences described in the proposed guidelines. | ||
==== Fonts ==== | |||
'''Status:''' guidelines approved | |||
Now that we're going to make many of the free fonts in the distribution available over the Internet, we'd like to take the opportunity to make sure the licensing metadata is populated in each of them so we can be sure we're complying with their licenses. | |||
There are [http://patches.fedorapeople.org/js-repoqueries/fonts-neither 80 fonts in 28 packages] that are currently missing the necessary data. Additionally, the remainder will need to be audited to make sure they actually specify correct license data. (This will be mostly scripted away so it won't be too onerous.) | |||
Finally, [[User:Patches/PackagingDrafts/FontsPolicy new guidelines]] will be put into effect to ensure this is done correctly in the future. | |||
==== Flash ==== | |||
'''Status:''' [https://bugzilla.redhat.com/show_bug.cgi?id=1000236 bugs filed] | |||
It seems [[Web Assets/Flash/Packages|a number of packages]] contain precompiled Flash files. [[Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries|This has never been acceptable for Fedora.]] These packages will be identified and bugs will be filed as part of the cleanup tasks. | |||
==== FPC ==== | |||
'''Status:''' [[Packaging:Web Assets|guidelines approved]] | |||
The Fedora Packaging Committee will need to consider and approve drafts pertaining to [[User:Patches/PackagingDrafts/Web Assets|Web Assets]] and [[User:Patches/PackagingDrafts/JavaScript|JavaScript]]. | |||
==== FPC/FESCo ==== | |||
'''Status:''' [https://lists.fedoraproject.org/pipermail/devel/2013-August/187836.html effectively already done?] | |||
FPC/FESCo may or may not want to consider a firm date for sunsetting the bundled JavaScript exception and requiring all new packages to meet the new guidelines. | |||
=== Fedora 21 === | |||
This section describes tasks that are scheduled to be completed during the Fedora 21 release cycle. | |||
==== The web-assets RPM ==== | |||
=== <code>httpd</code> === | A <code>web-assets-httpd</code> package will make the web assets directory available at the final decided location. | ||
==== <code>httpd</code> ==== | |||
<code>httpd</code> may want to add <code>Requires: web-assets-httpd</code> (or ship the configuration itself, though [https://lists.fedoraproject.org/pipermail/packaging/2013-July/009313.html it was suggested that it would be better seperate]) to make the assets directory available unconditionally by default. This will simplify packaging. (We really would prefer that JavaScript libraries not depend on httpd, as they could be used by Node.js or Ruby.) | <code>httpd</code> may want to add <code>Requires: web-assets-httpd</code> (or ship the configuration itself, though [https://lists.fedoraproject.org/pipermail/packaging/2013-July/009313.html it was suggested that it would be better seperate]) to make the assets directory available unconditionally by default. This will simplify packaging. (We really would prefer that JavaScript libraries not depend on httpd, as they could be used by Node.js or Ruby.) | ||
=== | ==== FPC ==== | ||
FPC approved an initial guidelines draft during the Fedora 20 release cycle. Several issues were identified during the discussions of the F20 feature and guidelines both within Fedora and with other distributions (namely Debian) that will require amendments to the guidelines, namely: | |||
* official guidelines for replacing directories with symlinks [https://fedorahosted.org/fpc/ticket/385 (fpc ticket #385)] | |||
* provide general recommendations for directory structure to avoid future pain, given Panu's comments in the above ticket | |||
* switch http daemon path to /.sysassets to prevent it from appearing in directory listings if implemented as a symlink | |||
* generalize the static inclusion guidelines to be useful for static linking and use plain Provides and not a macro | |||
==== other HTTP daemons ==== | |||
It would be nice to support additional HTTP daemons as well. I'll investigate whether we can support them with drop-in configuration files or symlinks from other <code>web-assets</code> subpackages in addition to httpd. | |||
==== Fonts ==== | |||
Now that we're going to make many of the free fonts in the distribution available over the Internet, we'd like to take the opportunity to make sure the licensing metadata is populated in each of them so we can be sure we're complying with their licenses. New guidelines were implemented during the Fedora 20 release cycle to prevent this from occurring in the future. | |||
There are [http://patches.fedorapeople.org/js-repoqueries/fonts-neither 80 fonts in 28 packages] that are currently missing the necessary data. Additionally, the remainder will need to be audited to make sure they actually specify correct license data. (This will be mostly scripted away so it won't be too onerous.) | |||
==== Flash ==== | |||
It seems [[Web Assets/Flash/Packages|a number of packages]] contain precompiled Flash files. [[Packaging:Guidelines#No_inclusion_of_pre-built_binaries_or_libraries|This has never been acceptable for Fedora.]] | |||
During the Fedora 20 release cycle, [https://bugzilla.redhat.com/show_bug.cgi?id=1000236 bugs were filed against packages that bundle these files]. Many were fixed, some are still left to do. Work on this will continue during the Fedora 21 release cycle. | |||
=== Foreign languages === | ==== Foreign languages ==== | ||
Many foreign languages support JavaScript, like Ruby via <code>rubygem-execjs</code> or Java via <code>rhino</code>. Several of them already depend on existing JavaScript library packages (virtually all node.js-related ones) and will need to be modified to conform to the new locations specified by the guidelines. | Many foreign languages support JavaScript, like Ruby via <code>rubygem-execjs</code> or Java via <code>rhino</code>. Several of them already depend on existing JavaScript library packages (virtually all node.js-related ones) and will need to be modified to conform to the new locations specified by the guidelines. | ||
=== Node.js === | ==== Node.js ==== | ||
Several node packages ship JavaScript for the browser and will need to be modified to conform to the new guidelines. | Several node packages ship JavaScript for the browser and will need to be modified to conform to the new guidelines. | ||
=== | === Handwavy Future === | ||
==== JavaScript packagers ==== | |||
Some longstanding reviews (like jquery) can now be completed with clear guidelines, paving the way for web applications to start being migrating to using proper dependencies instead of bundling. Several will be targeted for the Fedora 21 release in separate Change requests. | |||
All web applications currently in the distribution should be examined for bundled JavaScript, which should be packaged separately. | |||
=== | ==== Web application packagers ==== | ||
Web applications can migrate to new JavaScript/Web Asset packages as they become available. | |||
There are a lot of potentially affected packages. There's no hope of fixing them all in one release cycle, and even the dependency chain for jQuery, a big ticket package, could take several months to get reviewed and imported. Therefore, this Change proposal just seeks FESCo/FPC approval for a start to unravelling this madness. This will be a long transition, and will probably make the systemd transition look like a day in the park. | |||
Just to give you an idea of the magnitude: there are [http://patches.fedorapeople.org/js-repoqueries/js.txt 1836 packages] that ship <code>.js</code> files, discounting Node.js packages. (Though those certainly all don't necessarily include bundled JS.) There's a lot of low-hanging fruit: for instance, there are [http://patches.fedorapeople.org/js-repoqueries/jquery.js.txt 963 packages] that ship a file called <code>jquery.js</code>, a whole bunch of which is just included as output from documentation generators like Sphinx or rdoc. (Though keep in mind that isn't necessarily the only JS these packages bundle.) More low hanging fruit can be gleamed from [http://patches.fedorapeople.org/js-repoqueries/js-frequency.txt this list of frequencies of filenames ending in <code>.js</code>]. | |||
== Upgrade/compatibility impact == | == Upgrade/compatibility impact == | ||
| Line 109: | Line 158: | ||
== How To Test == | == How To Test == | ||
* Make sure the directories exist. | * Make sure the directories exist. | ||
* Install <code>httpd</code> and <code>web-assets-httpd</code>, then verify that you can navigate to <code>http://localhost/ | * Install <code>httpd</code> and <code>web-assets-httpd</code>, then verify that you can navigate to <code>http://localhost/.sysassets/</code> and <code>http://localhost/.sysassets/javascript/</code>. | ||
== User Experience == | == User Experience == | ||
| Line 116: | Line 165: | ||
== Dependencies == | == Dependencies == | ||
* Requires added to <code>httpd</code> | * Requires added to <code>httpd</code> | ||
* nodejs packages affected: | * nodejs packages affected: | ||
** uglify-js | ** uglify-js | ||
| Line 148: | Line 193: | ||
== Release Notes == | == Release Notes == | ||
Fedora | Fedora 21 introduces standard directories for web assets (such as web user interface libraries) in /usr/share/web-assets, and JavaScript code in <code>/usr/share/javascript</code>. In future releases, web applications will be migrated to using shared components, eliminating the need for security updates to the same code in multiple packages. | ||
[[Category: | [[Category:ChangePageIncomplete]] | ||
<!-- When your change proposal page is completed and ready for review and announcement --> | <!-- When your change proposal page is completed and ready for review and announcement --> | ||
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | ||
Latest revision as of 12:33, 16 December 2014
Web Assets
Summary
Traditionally, Fedora has been pushing bits from its various servers to people's browsers in an ad-hoc fashion, and issues surrounding JavaScript have been swept under the rug. This change proposal provides a simple framework for shipping static web content and a way forward to treat JavaScript more closely to other code in the distribution.
Owner
- Name: T.C. Hollingsworth
- Email: tchollingsworth@gmail.com
Current status
- Targeted release: Fedora 21
- Last updated: 2014-01-18
- Tracker bug: F20WebAssets #998583
Detailed Description
A standard directory (/usr/share/web-assets) for static bits that are intended to be delivered to web browsers, such as CSS Frameworks, UI libraries, etc. will be introduced. HTTP daemons in the distribution should make this directory available publicly as /.sysassets.
Additionally, a standard directory (/usr/share/javascript) for JavaScript code, which may be used in browsers or server-side by applications such as nodejs or rubygem-execjs will also be introduced.
Finally, new packaging guidelines will provide instructions for making proper use of these new directories.
Benefit to Fedora
Packagers will benefit from clear guidelines for shipping JavaScript code and other modular content that can be used by multiple web applications.
By eliminating bundling of JavaScript code within web applications and replacing it with proper packaging, the security footprint of the distribution will be improved.
Scope
Fedora 20
This section describes tasks already completed during the Fedora 20 release cycle.
The web-assets RPM
Status: built
A web-assets package will be introduced. A preliminary SRPM is available here.
The web-assets-filesystem subpackage will be provided, which contains the following directories:
/usr/share/web-assets/usr/share/javascript
Additionally the following symlinks will be provided:
/usr/share/web-assets/javascriptpoints to/usr/share/javascript/usr/share/web-assets/fontspoints to/usr/share/fonts(so any Fedora font package can be used as a web font)
A web-assets-devel subpackage will be provided that provides macros like %{_assetdir} and other conveniences described in the proposed guidelines.
Fonts
Status: guidelines approved
Now that we're going to make many of the free fonts in the distribution available over the Internet, we'd like to take the opportunity to make sure the licensing metadata is populated in each of them so we can be sure we're complying with their licenses.
There are 80 fonts in 28 packages that are currently missing the necessary data. Additionally, the remainder will need to be audited to make sure they actually specify correct license data. (This will be mostly scripted away so it won't be too onerous.)
Finally, User:Patches/PackagingDrafts/FontsPolicy new guidelines will be put into effect to ensure this is done correctly in the future.
Flash
Status: bugs filed
It seems a number of packages contain precompiled Flash files. This has never been acceptable for Fedora. These packages will be identified and bugs will be filed as part of the cleanup tasks.
FPC
Status: guidelines approved
The Fedora Packaging Committee will need to consider and approve drafts pertaining to Web Assets and JavaScript.
FPC/FESCo
Status: effectively already done?
FPC/FESCo may or may not want to consider a firm date for sunsetting the bundled JavaScript exception and requiring all new packages to meet the new guidelines.
Fedora 21
This section describes tasks that are scheduled to be completed during the Fedora 21 release cycle.
The web-assets RPM
A web-assets-httpd package will make the web assets directory available at the final decided location.
httpd
httpd may want to add Requires: web-assets-httpd (or ship the configuration itself, though it was suggested that it would be better seperate) to make the assets directory available unconditionally by default. This will simplify packaging. (We really would prefer that JavaScript libraries not depend on httpd, as they could be used by Node.js or Ruby.)
FPC
FPC approved an initial guidelines draft during the Fedora 20 release cycle. Several issues were identified during the discussions of the F20 feature and guidelines both within Fedora and with other distributions (namely Debian) that will require amendments to the guidelines, namely:
- official guidelines for replacing directories with symlinks (fpc ticket #385)
- provide general recommendations for directory structure to avoid future pain, given Panu's comments in the above ticket
- switch http daemon path to /.sysassets to prevent it from appearing in directory listings if implemented as a symlink
- generalize the static inclusion guidelines to be useful for static linking and use plain Provides and not a macro
other HTTP daemons
It would be nice to support additional HTTP daemons as well. I'll investigate whether we can support them with drop-in configuration files or symlinks from other web-assets subpackages in addition to httpd.
Fonts
Now that we're going to make many of the free fonts in the distribution available over the Internet, we'd like to take the opportunity to make sure the licensing metadata is populated in each of them so we can be sure we're complying with their licenses. New guidelines were implemented during the Fedora 20 release cycle to prevent this from occurring in the future.
There are 80 fonts in 28 packages that are currently missing the necessary data. Additionally, the remainder will need to be audited to make sure they actually specify correct license data. (This will be mostly scripted away so it won't be too onerous.)
Flash
It seems a number of packages contain precompiled Flash files. This has never been acceptable for Fedora.
During the Fedora 20 release cycle, bugs were filed against packages that bundle these files. Many were fixed, some are still left to do. Work on this will continue during the Fedora 21 release cycle.
Foreign languages
Many foreign languages support JavaScript, like Ruby via rubygem-execjs or Java via rhino. Several of them already depend on existing JavaScript library packages (virtually all node.js-related ones) and will need to be modified to conform to the new locations specified by the guidelines.
Node.js
Several node packages ship JavaScript for the browser and will need to be modified to conform to the new guidelines.
Handwavy Future
JavaScript packagers
Some longstanding reviews (like jquery) can now be completed with clear guidelines, paving the way for web applications to start being migrating to using proper dependencies instead of bundling. Several will be targeted for the Fedora 21 release in separate Change requests.
All web applications currently in the distribution should be examined for bundled JavaScript, which should be packaged separately.
Web application packagers
Web applications can migrate to new JavaScript/Web Asset packages as they become available.
There are a lot of potentially affected packages. There's no hope of fixing them all in one release cycle, and even the dependency chain for jQuery, a big ticket package, could take several months to get reviewed and imported. Therefore, this Change proposal just seeks FESCo/FPC approval for a start to unravelling this madness. This will be a long transition, and will probably make the systemd transition look like a day in the park.
Just to give you an idea of the magnitude: there are 1836 packages that ship .js files, discounting Node.js packages. (Though those certainly all don't necessarily include bundled JS.) There's a lot of low-hanging fruit: for instance, there are 963 packages that ship a file called jquery.js, a whole bunch of which is just included as output from documentation generators like Sphinx or rdoc. (Though keep in mind that isn't necessarily the only JS these packages bundle.) More low hanging fruit can be gleamed from this list of frequencies of filenames ending in .js.
Upgrade/compatibility impact
None at this time.
Web Applications migrating their dependencies to the new framework will need to take care to provide a sane upgrade path, but that should be relatively simple.
How To Test
- Make sure the directories exist.
- Install
httpdandweb-assets-httpd, then verify that you can navigate tohttp://localhost/.sysassets/andhttp://localhost/.sysassets/javascript/.
User Experience
Web Developers who want to use these libraries and frameworks in their applications will have a single place to look going forward.
Dependencies
- Requires added to
httpd - nodejs packages affected:
- uglify-js
- uglify-js1
- coffee-script
- nodejs-less
- Ruby packages affected:
- rubygem-coffee-script
- rubygem-coffee-script-source
- rubygem-uglifier
FAQ
Using /usr/share/javascript allows for consistency with other distributions (like Debian) and reflects the fact that JavaScript code can be run on Fedora itself as well as being pushed to the browser.
Why not just ship Apache configuration files for this kind of stuff like we've been doing/has been suggested before?
That leaves other http daemons and servers that don't involve traditional HTTP daemons (like many Node.js applications) hung out to dry. By just using a single, standard directory, and making that accessible from all HTTP daemons, support for many HTTP daemons is provided, and the need for packagers to keep up with changing Apache configuration formats just to ship a few simple JS files is eliminated.
Contingency Plan
None needed, either the directories get added or they don't.
Documentation
The new Packaging Guidelines referenced above.
Release Notes
Fedora 21 introduces standard directories for web assets (such as web user interface libraries) in /usr/share/web-assets, and JavaScript code in /usr/share/javascript. In future releases, web applications will be migrated to using shared components, eliminating the need for security updates to the same code in multiple packages.