| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
GOP has two goals:
| 14.5.1 Motivation | ||
| 14.5.2 Ongoing jobs | ||
| 14.5.3 Policy decisions | ||
| 14.5.4 Policy decisions (finished) |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Most readers are probably familiar with the LilyPond Grand Documentation Project, which ran from Aug 2007 to Aug 2008. This project involved over 20 people and resulted in an almost complete rewrite of the documentation. Most of those contributors were normal users who decided to volunteer their time and effort to improve lilypond for everybody. By any measure, it was a great success.
The Grand Organization Project aims to do the same thing with a larger scope – instead of focusing purely on documentation, the project aims to improve all parts of LilyPond and its community. Just as with GDP, the main goal is to encourage and train users to become more involved.
If you have never contributed to an open-source project before – especially if you use Windows or OSX and do not know how to program or compile programs – you may be wondering if there’s anything you can do. Rest assured that you can help.
One of the reasons I’m organizing GOP is "trickle-up" development. The idea is this: doing easy tasks frees up advanced developers to do harder tasks. Don’t ask "am I the best person for this job"; instead, ask "am I capable of doing this job, so that the current person can do stuff I can’t do?".
For example, consider lilypond’s poor handling of grace notes in conjunction with clef and tempo changes. Fixing this will require a fair amount of code rewriting, and would take an advanced developer a few weeks to do. It’s clearly beyond the scope of a normal user, so we might as well sit back and do nothing, right?
No; we can help, indirectly. Suppose that our normal user starts answering more emails on lilypond-user. This in turn means that documentation writers don’t need to answer those emails, so they can spend more time improving the docs. I’ve noticed that all doc writers tackle harder and harder subjects, and when they start writing docs on scheme programming and advanced tweaks, they start contributing bug fixes to lilypond. Having people performing these easy-to-moderate bug fixes frees up the advanced developers to work on the really hard stuff... like rewriting the grace note code.
Having 1 more normal user answering emails on lilypond-user won’t have a dramatic ‘trickle-up’ effect all by itself, of course. But if we had 8 users volunteering to answer emails, 6 users starting to write documentation, and 2 users editing LSR... well, that would free up a lot of current bug-fixing-capable contributors to focus on that, and we could start to make a real dent in the number of bugs in lilypond. Quite apart from the eased workload, having that many new helpers will provide a great moral boost!
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Although GOP is a short-term project, the main goal is to train more people to handle ongoing jobs. The more people doing these jobs, the lighter the work will be, and the more we can get done with lilypond!
Also, it would be nice if we had at least one "replacement" / "understudy" for each role – too many tasks are only being done by one person, so if that person goes on vacation or gets very busy with other matters, work in that area grinds to a halt.
Technical requirements: ability to run LilyDev.
Technical requirements: use of a web browser. LilyPond requirements: you should be familiar with most of Notation chapters 1 and 2 (or be willing to read the docs to find out).
Technical requirements: development environment (such as LilyDev), ability to read+write scheme and/or C++ code.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
There are a number of policy decisions – some of them fairly important – which we have been postponing for a few years. We are now discussing them slowly and thoroughly; agenda and exact proposals are online:
https://lilypond.org/~graham/gop/index.html |
Below is a list of policies which are not “on the agenda” yet.
Note that the presence of an item on this list does not mean that everybody thinks that something needs to be done. Inclusion in this simply means that one developer thinks that we should discuss it. We are not going to filter this list; if any developer thinks we should discuss something, just add it to the bottom of the list. (the list is unsorted)
As GOP progresses, items from this list will be put on the agenda and removed from this list. I generally try to have one month’s discussion planned in advance, but I may shuffle things around to respond to any immediate problems in the developer community.
There are some item(s) not displayed here; these are questions that were posed to me privately, and I do not feel justified in discussing them publicly without the consent of the person(s) that brought them up. They will initially be discussed privately on the lilypond-hackers mailing list – but the first question will be "do we absolutely need to do this privately", and if not, the discussion will take place on lilypond-devel like the other items.
In most policy discussions in lilypond over the past few years, the first half (or more) is wasted arguing on the basis of incorrect or incomplete data; once all the relevant facts are brought to light, the argument is generally resolved fairly quickly. In order to keep the GOP discussions focused, each topic will be introduced with a collection of relevant facts and/or proposals. It is, of course, impossible to predict exactly which facts will be relevant to the discussion – but spending an hour or two collecting information could still save hours of discussion.
Note: The estimated time required for "prep work", and the following discussion, has been added to each item. At the moment, there is an estimated 30 hours of prep work and 140 hours of discussion.
(prep: 2 hours. discuss: 10 hours)
(prep: 2 hours. discuss: 5 hours)
(prep: 1 hour. discuss: 5 hours)
(prep: 5 hours. discuss: 15 hours)
(prep: 2 hours. discuss: 10 hours)
Rationale: - aids maintainability of code base - "second reading" so newer developers can look up unfamiliar stuff - will help to keep things simple, even if the code is doing complex stuff discourages "secret squirrel" coding, e.g. "how much functionality can I squeeze into as few characters as possible" "comments are for wimps" - will aid not *discouraging* new developers to join the project
(prep: 2 hours. discuss: 10 hours)
(prep: 5 hours. discuss: 15 hours)
(prep: 1 hour. discuss: 5 hours)
Probably not worth making this a weekly thing on its own, but we can probably wrap it up with some other code-related questions.
(prep: 15 minutes. discuss: 3 hours)
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Here is a record the final decisions, along with links to the discussions.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
We will follow the indentation described in PEP-8. http://www.python.org/dev/peps/pep-0008/
Once this is done, we should add python -tt to the build
system to avoid such errors in the future.
There should be absolutely no tab characters for indentation in
any .py file in lilypond git. All such files should be
converted to use spaces only.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Nothing much was decided. The list of responsibilities was slightly altered; see the new one in Mentors. We should encourage more use of the Frogs mailing list. There’s a list of contributor-mentor pairs in:
https://github.com/gperciva/lilypond-extra/blob/master/people/mentors.txt |
That’s pretty much it.
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00311.html |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Speaking academically, C++ code style is a "solved problem". Let’s pick one of the existing solutions, and let a computer deal with this. Humans should not waste their time, energy, and creativity manually adding tabs or spaces to source code.
We have modified fixcc.py to use astyle, along with extra
regex tweaks.
fixcc.py will not be considered grounds for
rejecting a patch.
fixcc.py are pushed. From time to time, Graham will run
the formatter on the entire code base, and commit the resulting
changes.
In a few months, we will tighten up this policy item (with some sort of automatic processing), but that is outside the scope of this policy item and is a matter for later discussion.
LilyPond is a GNU project, so it makes sense to follow the GNU coding standards. These standards state:
We don’t think of these recommendations as requirements, because it causes no problems for users if two different programs have different formatting styles.
But whatever style you use, please use it consistently, since a mixture of styles within one program tends to look ugly. If you are contributing changes to an existing program, please follow the style of that program.
(http://www.gnu.org/prep/standards/html_node/Formatting.html)
With that in mind, we do not think that we must blindly follow the formatting given by the currrent version of Emacs.
We can avoid some of the style change pollution in git history by ignoring whitespaces changes:
git diff -w |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
A brief history of releases:
| date (YYYY-MM-DD) | version | comment |
|---|---|---|
| 2008-10-28 | 2.11.63 | nobody checking regtests |
| 2008-11-17 | 2.11.64 | |
| 2008-11-29 | 2.11.65 | |
| 2008-12-23 | 2.12.0 | |
| 2009-01-01 | somewhere around here, Graham becomes officially release manager, but Han-Wen still builds the actual releases | |
| 2009-01-01 | 2.12.1 | |
| 2009-01-25 | 2.12.2 | |
| 2009-02-28 | 2.13.0 | |
| 2009-06-01 | 2.13.1 | note jump in time! |
| 2009-06-27 | 2.13.2 | first Graham release? |
| 2009-07-03 | 2.13.3 | |
| 2009-09-09 | Graham arrives in Glasgow, gets a powerful desktop computer, and begins serious work on GUB (sending bug reports to Jan). It takes approximately 100 hours until GUB is stable enough to make regular releases. | |
| 2009-09-24 | 2.13.4 | |
| 2009-10-02 | 2.13.5 | |
| 2009-10-22 | 2.13.6 | |
| 2009-11-05 | 2.13.7 | |
| ... | ||
| 2010-01-13 | 2.12.3 | |
| ... | ||
| 2010-03-19 | 2.13.16 | Bug squad starts doing a few regtest comparisons, but IIRC the effort dies out after a few weeks (BLUE) |
| ... | ||
| 2010-08-04 | 2.13.29 | Phil starts checking regtests (BLUE) |
| ... | ||
| 2011-01-12 | 2.13.46 | release candidate 1 (GREEN) |
| ... | ||
| 2011-05-30 | 2.13.63 | release candidate 7 (GREEN) |
| 2011-06-06 | 2.14.0 |
A ‘csv’ spreadsheet is available.
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html |
lilypond-issues-analysis.csv lilypond-issues-analysis-trim-duplicates.csv |
There 148 issues marked with Priority=Critical in the tracker.
I’ve done an analysis, and it looks to me like there was initially a backlog of critical issues that weren’t fixed, and little work was being done to eliminate critical issues.
Somewhere about 2010-08-01, critical issues started to disappear, but occasional new ones appeared.
There were a couple of major changes that introduced unanticipated regressions (new spacing code, beam collision avoidance). These produced more than the expected number of regressions.
It appears to me that we didn’t really get serious about eliminating critical bugs until about 2010-06-15 or so. After that point, the number of critical bugs more-or-less steadily decreased until we got to a release candidate.
Of particular interest, the first release candidate of 2.14 was released on 2011-01-12. Over the next 10 days, about a dozen bugs were reported and fixed. Release candidate 2 came out on 2011-02-09. No surge of bugs occurred with this release. Candidate 3 came out on 2011-03-13; we got 2 bugs per week. Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came out on 2011-04-07. We got a couple of bugs per week.
Han-Wen: Overall, I think this cycle took too long Mike: I agree Graham: +1 |
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This proposal was too broad; after a month of discussion, Graham withdrew the proposal. Portions of it will be introduced in later proposals.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Potentially sensitive or private matters will be referred to Graham. He will then decide who should discuss the matter on an ad-hoc basis, and forward or CC them on future emails.
For emphasis, the project administrators are Han-Wen, Jan, and Graham; those three will always be CC’d on any important discussions.
The lilypond-hackers mailing list will be removed.
There is some unhappy history about this idea in our development community:
The idea of private mailing lists is hardly uncommon in open-source software. For example,
http://lwn.net/Articles/394660/ about debian-private http://subversion.apache.org/mailing-lists.html private@ http://www.freebsd.org/administration.html#t-core http://foundation.gnome.org/legal/ board members pledge to keep certain matters confidential every security team of every GNU/Linux distribution and OS |
In fact, Karl Fogel’s “Producing Open Source Software” explicitly suggests a private mailing list for some circumstances:
[on granting commit/push access to a contributor] But here is one of the rare instances where secrecy is appropriate. You can't have votes about potential committers posted to a public mailing list, because the candidate's feelings (and reputation) could be hurt. http://producingoss.com/en/consensus-democracy.html#electorate |
Many projects have an official board of directors, or a list of “core developers”, with set term limits and elections and stuff.
I don’t think that we’re that big. I think we’re still small enough, and there’s enough trust and consensus decisions, that we can avoid that. I would rather that we kept on going with trust+consensus for at least the next 2-3 years, and spent more time+energy on bug fixes and new features instead of administrative stuff.
Project administrators are Han-Wen, Jan, and Graham.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
We shall treat developers (and contributors) as Independent volunteers: each person does whatever they want, whenever they want. We have busy careers and lives; we make no expectations of action from anybody (with the exception of the 6 people in “Meister” positions).
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
We will delete the “priority” field of the issue tracker altogether. The “type” system will be tweaked.
Type-critical:
make or make
doc, from an empty build tree, in a first run, if
configure does not report any errors.
To limit this scope of this point, we will assume that the contributor is using the latest LilyDev and has read the relevant part(s) of the Contributor’s Guide. Problems in other chapters of the CG are not sufficient to qualify as Type-Critical.
Unless otherwise specified, the current types and labels will continue to be used. The new types introduced by this proposal are:
A new label will be added:
We can remind users that they can “star” an issue to indicate that they care about it. Since we resolved to treat developers as independent volunteers, there is no expectation that anybody will look at those stars, but if any developer want to organize their work schedule according to the stars, they are welcome to do so.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
If there are build problems, then it should be easier to find out why it’s failing. This will be achieved with log files, as well as possibly including scripts which automatically display portions of those log files for a failing build.
We will also add targets for building a specific manual (for quick+easy checking of doc work), as well as for building all documentation in a specific language (either English or a translated language).
When you run make doc,
make(1).
Note that make(1) refers to a specific executable file on
unix computers, and is not a general term for the build system.
The user may optionally request additional output to be printed;
this is controlled with the VERBOSE=x flag. In such cases,
all output will still be written to log files; the console output
is strictly additional to the log files.
A single make doc will therefore result in hundreds of log
files. Log files produced from individual lilypond runs are not
under our control; apart from that, I anticipate having one or two
dozen log files. As long as it is clear which log file is
associated with which operation(s), I think this is entirely
appropriate. The precise implementation will be discussed for
specific patches as they appear.
*.log. The order
of lines from these streams should be preserved.
make --silent, a non-failing
build should print absolutely nothing to the screen.
If this proposal is accepted, none of these policies will be assumed to apply to any other aspect of the build system. Policies for any other aspect of the build system will be discussed in separate proposals.
However, there is a danger in this approach, that vital error messages can also be lost, thus preventing the cause of the failure of a make being found. We therefore need to be exceptionally careful to move cautiously, include plenty of tests, and give time for people to experiment/find problems in each stage before proceeding to the next stage.
This will be done by starting from individual lilypond calls within lilypond-book, and slowly moving to “larger” targets of the build system – after the individual lilypond calls are are producing the appropriate amount of output and this is saved in the right place and we can automatically isolate parts of a failing build, we will work on lilypond-book in general, and only then will we look at the build system itself.
There is an existing make variable QUIET_BUILD, which alter the amount of output being displayed (https://lilypond.org/doc/v2.15/Documentation/contributor/useful-make-variables ). We are not planning on keeping this make variable.
The standard way for GNU packages to give more output is with a
V=x option. Presumably this is done by increasing
x? If we support this option, we should still write log
files; we would simply print more of the info in those log files
to screen.
The command tee may be useful to write to a file and
display to stdout (in the case of VERBOSE).
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] |
This document was generated by gub on July 13, 2020 using texi2html 1.82.