(Initial page creation, based on Features/StaticAnalysisOfCPythonExtensions) |
(User Experience: clarify how to install and run the examples) |
||
(26 intermediate revisions by 2 users not shown) | |||
Line 8: | Line 8: | ||
== Summary == | == Summary == | ||
<!-- A sentence or two summarizing what this feature is and what it will do. This information is used for the overall feature summary page for each release. --> | <!-- A sentence or two summarizing what this feature is and what it will do. This information is used for the overall feature summary page for each release. --> | ||
I have created GCC | I have created a GCC plugin that [https://fedorahosted.org/gcc-python-plugin/ embeds Python within GCC]. Interested developers should be able to use this to easily extend GCC by writing Python scripts, such as adding new warnings for a library that they work on, without needing to use C. | ||
== Owner == | == Owner == | ||
Line 18: | Line 18: | ||
== Current status == | == Current status == | ||
* Targeted release: [[Releases/ | * Targeted release: [[Releases/16 | Fedora 16 ]] | ||
* Last updated: 2011- | * Last updated: 2011-09-19 | ||
* Percentage of completion: | * Percentage of completion: 95% | ||
Packaged and built within Fedora as gcc-python-plugin-0.3-1.fc16 (review was {{bz|725570}}) | |||
The latest tarball release I've made is 0.6, and there's as updated build to that version awaiting testing, to go into the F16 tree: https://admin.fedoraproject.org/updates/gcc-python-plugin-0.6-2.fc16 | |||
(this fixes some segfaults seen on i686 (fixed in 0.4), along with numerous other improvements). | |||
However, that build currently fails to work, with this error: https://bugzilla.redhat.com/show_bug.cgi?id=739766 | |||
<!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. --> | <!-- CHANGE THE "FedoraVersion" TEMPLATES ABOVE TO PLAIN NUMBERS WHEN YOU COMPLETE YOUR PAGE. --> | ||
Line 46: | Line 55: | ||
* as above, but for the debug builds of Python 2 and Python 3, rather than the optimized builds. | * as above, but for the debug builds of Python 2 and Python 3, rather than the optimized builds. | ||
The "upstream" for the plugins | The "upstream" for the plugins is this Fedora-hosted project: https://fedorahosted.org/gcc-python-plugin/ | ||
== Benefit to Fedora == | == Benefit to Fedora == | ||
Line 74: | Line 83: | ||
3. What are the expected results of those actions? | 3. What are the expected results of those actions? | ||
--> | --> | ||
The plugin already has a testsuite, which "make" runs, and is run when the rpm is built (within the %check) | |||
The plugin ships with some sample scripts (within the -docs subpackage, below /usr/share/doc/gcc-python-plugin-docs-*/examples) | |||
* show-gimple.py | |||
* show-ssa.py | |||
which halt the compilation at each function, showing a graphviz-rendered diagram of that function (in the "gimple" representation, or the gimple representation after transforming to Static Single Assigment form). An interested developer can invoke these scripts by replacing "gcc" with "gcc-with-python2 PATH_TO_SCRIPT": | |||
<code>gcc-with-python2 /usr/share/doc/gcc-python-plugin-docs-*/examples/show-gimple.py main.c</code> | |||
== User Experience == | == User Experience == | ||
Line 79: | Line 97: | ||
Non-technical end-users of Fedora should see no difference. | Non-technical end-users of Fedora should see no difference. | ||
Developers will be able to invoke Python scripts whilst running GCC by | Developers will be able to invoke Python scripts whilst running GCC by using a script that invokes gcc with the plugin. | ||
For Python 2, the plugin is in the <code>gcc-python2-plugin</code> package: | |||
<pre> | |||
# Install the plugin ("gcc-python2-plugin") | |||
$ sudo yum install gcc-python2-plugin | |||
</pre> | |||
and can be run whilst compiling using: | |||
<pre> | |||
$ gcc-with-python2 PATH_TO_SCRIPT.py rest of regular gcc args | |||
</pre> | |||
The examples are in gcc-python-plugin-docs: | |||
<pre> | |||
# Install the docs and examples: | |||
$ sudo yum install gcc-python-plugin-docs | |||
# Show GCC's internal representation when compiling main.c: | |||
$ gcc-with-python2 /usr/share/doc/gcc-python-plugin-docs-0.6/examples/show-gimple.py main.c | |||
</pre> | |||
For Python 3, the plugin is within the <code>gcc-python3-plugin</code> package: | |||
<pre> | |||
$ sudo yum install gcc-python3-plugin | |||
$ gcc-with-python3 PATH_TO_SCRIPT.py rest of regular gcc args | |||
</pre> | |||
Alternatively, this is equivalent to gcc-with-python2 and gcc-with-python3 | |||
<pre> | |||
# Python 2: | |||
$ gcc -fplugin=python2 -fplugin-arg-python2-script=PATH_TO_SCRIPT.py | |||
# Python 3: | |||
$ gcc -fplugin=python3 -fplugin-arg-python3-script=PATH_TO_SCRIPT.py | |||
</pre> | |||
== Dependencies == | == Dependencies == | ||
Line 95: | Line 144: | ||
== Contingency Plan == | == Contingency Plan == | ||
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "None necessary, revert to previous release behaviour." Or it might not. If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy. --> | <!-- If you cannot complete your feature by the final development freeze, what is the backup plan? This might be as simple as "None necessary, revert to previous release behaviour." Or it might not. If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy. --> | ||
* | * We can simply remove the plugin. This would impact [[Features/StaticAnalysisOfCPythonExtensions]]; see the contingency plan for that feature. | ||
== Documentation == | == Documentation == | ||
<!-- Is there upstream documentation on this feature, or notes you have written yourself? Link to that material here so other interested developers can get involved. --> | <!-- Is there upstream documentation on this feature, or notes you have written yourself? Link to that material here so other interested developers can get involved. --> | ||
API documentation for the plugin can be seen at http://readthedocs.org/docs/gcc-python-plugin/en/latest/index.html | |||
== Release Notes == | == Release Notes == | ||
<!-- The Fedora Release Notes inform end-users about what is new in the release. Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ --> | <!-- The Fedora Release Notes inform end-users about what is new in the release. Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ --> | ||
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns. If there are any such changes involved in this feature, indicate them here. You can also link to upstream documentation if it satisfies this need. This information forms the basis of the release notes edited by the documentation team and shipped with the release. --> | <!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns. If there are any such changes involved in this feature, indicate them here. You can also link to upstream documentation if it satisfies this need. This information forms the basis of the release notes edited by the documentation team and shipped with the release. --> | ||
* GCC plugins that embed Python are now available. These were written by Fedora contributor David Malcolm | * GCC plugins that embed Python are now available, enabling developers to more easily hook into GCC's inner workings (e.g. to add new compiler warnings). These were written by Fedora contributor David Malcolm. | ||
== Comments and Discussion == | == Comments and Discussion == | ||
* See [[Talk:Features/ | * See [[Talk:Features/GccPythonPlugin]] <!-- This adds a link to the "discussion" tab associated with your page. This provides the ability to have ongoing comments or conversation without bogging down the main feature page --> | ||
[[Category: | [[Category:FeatureAcceptedF16]] | ||
<!-- When your feature page is completed and ready for review --> | <!-- When your feature page is completed and ready for review --> | ||
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler --> | <!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler --> | ||
<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete--> | <!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete--> | ||
<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process --> | <!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process --> | ||
[[Category:Python]] |
Latest revision as of 22:31, 4 October 2011
GCC Python Plugins
Summary
I have created a GCC plugin that embeds Python within GCC. Interested developers should be able to use this to easily extend GCC by writing Python scripts, such as adding new warnings for a library that they work on, without needing to use C.
Owner
- Name: Dave Malcolm
- Email: dmalcolm@redhat.com
Current status
- Targeted release: Fedora 16
- Last updated: 2011-09-19
- Percentage of completion: 95%
Packaged and built within Fedora as gcc-python-plugin-0.3-1.fc16 (review was RHBZ #725570)
The latest tarball release I've made is 0.6, and there's as updated build to that version awaiting testing, to go into the F16 tree: https://admin.fedoraproject.org/updates/gcc-python-plugin-0.6-2.fc16
(this fixes some segfaults seen on i686 (fixed in 0.4), along with numerous other improvements).
However, that build currently fails to work, with this error: https://bugzilla.redhat.com/show_bug.cgi?id=739766
Detailed Description
The plugins will allow interested developers to extend GCC in Python 2 and Python 3, without needing to deal with the C internals of GCC.
Example of potential uses include adding new compiler warnings based on:
- GTK's reference-counting semantics
- locking in the Linux kernel
- signal-safety in APIs.
- malloc/free usage (e.g. use-after-free bugs)
- array boundary checks
I aim to use the feature to add compile-time checking of the source code to all CPython extension modules in Fedora, as described in Features/StaticAnalysisOfCPythonExtensions
Other ideas include:
- visualizations of code structure
- semantic grepping of source code (e.g. "find all invocations of C++ methods named 'draw' that use the default params")
There will be four GCC plugins:
- a plugin embedding Python 2 (actually dynamically linked against libpython2.7.so)
- a plugin embedding python 3 (actually dynamically linked against libpython3.2.so)
- as above, but for the debug builds of Python 2 and Python 3, rather than the optimized builds.
The "upstream" for the plugins is this Fedora-hosted project: https://fedorahosted.org/gcc-python-plugin/
Benefit to Fedora
Fedora is already a very attractive platform for software development. By making it very easy to extend GCC, the plugin makes Fedora even more compelling to developers. The plugin code was written by a Fedora developer, and is a "Fedora Hosted" project.
Scope
This involves:
- writing the plugin
- packaging it
- documenting it
- keeping it in sync with gcc
How To Test
The plugin already has a testsuite, which "make" runs, and is run when the rpm is built (within the %check)
The plugin ships with some sample scripts (within the -docs subpackage, below /usr/share/doc/gcc-python-plugin-docs-*/examples)
- show-gimple.py
- show-ssa.py
which halt the compilation at each function, showing a graphviz-rendered diagram of that function (in the "gimple" representation, or the gimple representation after transforming to Static Single Assigment form). An interested developer can invoke these scripts by replacing "gcc" with "gcc-with-python2 PATH_TO_SCRIPT":
gcc-with-python2 /usr/share/doc/gcc-python-plugin-docs-*/examples/show-gimple.py main.c
User Experience
Non-technical end-users of Fedora should see no difference.
Developers will be able to invoke Python scripts whilst running GCC by using a script that invokes gcc with the plugin.
For Python 2, the plugin is in the gcc-python2-plugin
package:
# Install the plugin ("gcc-python2-plugin") $ sudo yum install gcc-python2-plugin
and can be run whilst compiling using:
$ gcc-with-python2 PATH_TO_SCRIPT.py rest of regular gcc args
The examples are in gcc-python-plugin-docs:
# Install the docs and examples: $ sudo yum install gcc-python-plugin-docs # Show GCC's internal representation when compiling main.c: $ gcc-with-python2 /usr/share/doc/gcc-python-plugin-docs-0.6/examples/show-gimple.py main.c
For Python 3, the plugin is within the gcc-python3-plugin
package:
$ sudo yum install gcc-python3-plugin $ gcc-with-python3 PATH_TO_SCRIPT.py rest of regular gcc args
Alternatively, this is equivalent to gcc-with-python2 and gcc-with-python3
# Python 2: $ gcc -fplugin=python2 -fplugin-arg-python2-script=PATH_TO_SCRIPT.py # Python 3: $ gcc -fplugin=python3 -fplugin-arg-python3-script=PATH_TO_SCRIPT.py
Dependencies
This will require working closely with the gcc maintainer. I anticipate needing to rebuild the plugin each time that gcc is rebuilt.
Contingency Plan
- We can simply remove the plugin. This would impact Features/StaticAnalysisOfCPythonExtensions; see the contingency plan for that feature.
Documentation
API documentation for the plugin can be seen at http://readthedocs.org/docs/gcc-python-plugin/en/latest/index.html
Release Notes
- GCC plugins that embed Python are now available, enabling developers to more easily hook into GCC's inner workings (e.g. to add new compiler warnings). These were written by Fedora contributor David Malcolm.