Ticket #33 (closed defect: fixed)

Opened 4 years ago

Last modified 4 years ago

Make Mercurial trust jbu users

Reported by: rprogers Owned by: rprogers
Priority: major Milestone: First source release
Component: Trac Version: 0.1
Keywords: Cc: ajit@…
Blocked By: Blocking: #7
OS: All CPU: All
Sensitive:

Description (last modified by rprogers) (diff)

I think we should have a /etc/mercurial/hgrc.d/trust.rc file containing

[trusted]
groups = jbu
users = apache

to make it shut up about not trusting anyone. That sound right Ajit?

In particular, it would be nice to run the commit hook to update Trac without getting constant warning messages.

Change History

comment:1 Changed 4 years ago by ajit

Solution 1: To get rid of client side warnings about trusted (i.e., onhg push) every person pushing to the repository needs to add this trusted declaration to their $HOME/.hgrc file.

Solution 2: Add users = * to the trusted user set in ~apache/.hgrc, i.e., the web users' mercurial configuration file.

Solution 3: What Richard is proposing in this ticket, i.e., set a default trust on lungs. See Mercurial Trust.

The first solution is socially impractical. I don't think Solution 2 will work, as apache is likely not a regular user. Given these constraints, solution 3 is best.

comment:2 Changed 4 years ago by rprogers

  • Cc ajit@… added; ajit@… removed

comment:3 Changed 4 years ago by rprogers

After discussing this a bit with Ajit, I think there are 2 cases:

  1. Doing an hg push ssh://lungs/hg/gmtk the hook script is run as the pushing user, and the pushing user must trust the script. This is fixed by Ajit's solutions 1 or 3 above.
  2. Doing an hg push https://lungs/hg/gmtk the hook script is run by apache, so we just need to make apache trust the repo via something like solution 2 or 3.

It sounds like most people currently access the repo via ssh. If solution 3 was applied to all SSLI machines, all users running Mercurial on SSLI machines would trust all repos owned by the jbu group. Is it possible to apply solution 3 to just SSLI machines? Applying solution 3 to all EE machines might be undesirable, as it would force non-SSLI users to trust the jbu group. It also doesn't help for users pushing from non-SSLI machines.

Is solution 1 really impractical? It seems like Mercurial users do need to be aware of their ~/.hgrc files, so requesting they modify it occasionally might be reasonable. We could put an appropriate template in /etc/skel/.hgrc so that new users will trust the repo by default, though again we probably want to restrict that to SSLI machines.

Solution 2 might work nicely if we can convince people to access the repo via https instead of ssh.

I'm not sure any of these are ideal.

comment:4 Changed 4 years ago by rprogers

Most users cannot ssh into lungs, so for ssh access to the Hg repository they'd have to go through mouth or throat. Mouth & throat have the Hg repo filesystem at /g/j/repo/mercurial, so that works. Mouth & throat can also run /usr/nikola/pkgs/trac/bin/trac-admin. The problem is that the Trac environment directories are not currently available to mouth & throat, so they will not be able to execute the Hg commit hook. I'll check with Lee to see if it's possible to make the Trac environments available like the Hg repos, somewhere like /g/j/trac on mouth and thoat.

Of course, that means everyone on mouth & throat would be able to muck around with trac-admin, which would be dangerous. Solution 3 is looking better...

comment:5 Changed 4 years ago by rprogers

  • Description modified (diff)

comment:6 Changed 4 years ago by rprogers

After a couple hours of futzing around, I've figured out that to get the hook to work for HTTP access to the repository, we need to specify the hook in the hgweb.config file. It seems the CGI scripts completely ignore the repositories' .hg/hgrc file. It's likely we'd need different hooks (and possibly different config files) for each environment (GMTK & MSMS). We can configure the repository authentication the same way we have Trac's: SSLI users can use their username/password, others use their .htpasswd username/password.

To get the hook to work for SSH access to the repository, we would need to mount lungs:/projects/trac somewhere like {mouth,throat}:/g/j/trac and require all pushers to put [trusted] groups=jbu in their ~/.hgrc file. This would allow passwordless SSH access to the repostiory, but allows group jbu on mouth or throat full write access to the Trac environments. This is the same situation as for the Hg repositories, so in that sense it's no worse.

comment:7 Changed 4 years ago by rprogers

  • Blocking 7 added

(In #7) I think we're close to resolving the Mercurial permission and Trac integration issues (see #33). I'm also close to adding the Autotool work into CVS head (see #2). Jeff thinks the issues with Arthur's checkins are resolved (see #17) .Looking to complete the CVS -> Hg transition early next week. I will write up a HgHowTo covering basic tasks in Mercurial (see #46).

comment:8 Changed 4 years ago by rprogers

Lee has mounted lungs:/projects/trac at {mouth,throat}:/g/j/projects/trac, so I should be able to get the hook working for SSH access. HTTP access will require some coordination with Ajit.

comment:9 Changed 4 years ago by rprogers

mouth & throat need the MySQL client libraries to run the hook. I put a ticket into ssli-bugs to get them installed.

comment:10 Changed 4 years ago by rprogers

Need to change the Trac database connection string to mysql://tracuser:password@lungs/trac and then configure server to allow connections from mouth and throat.

comment:11 Changed 4 years ago by Richard Rogers <rprogers@…>

added space to test commit hook; refs #33
Last edited 4 years ago by rprogers (previous) (diff)

comment:12 Changed 4 years ago by rprogers

The above was added by a push from parrot using ssh://throat//g/j/repo/mercurial/repos/gmtk as the default repository. Here's what I saw:

$ hg push
running ssh throat "hg -R /g/j/repo/mercurial/repos/gmtk serve --stdio"
pushing to ssh://throat//g/j/repo/mercurial/repos/gmtk
searching for changes
1 changesets found
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
remote: running hook incoming: /g/j/projects/trac/hghook.sh
remote: /usr/nikola/pkgs/trac/lib/python2.6/site-packages/graphviz-0.7.6dev-py2.6.egg/graphviz/graphviz.py:23: DeprecationWarning: the sha module is deprecated; use the hashlib module instead
$ hostname
parrot.ee.washington.edu

It looks like all the issues are worked out for SSH access from mouth and throat. The deprecation warning is not a Mercurial issue (see ticket #45).

Still need to coordinate with Ajit on HTTP access.

comment:13 Changed 4 years ago by Ajit Singh <ajit@…>

just testing hook; refs #33
Last edited 4 years ago by rprogers (previous) (diff)

comment:14 Changed 4 years ago by rprogers

Just had a chat with Ajit on the HTTP access/configuration issues. Requirements we've identified so far:

  • Ability to access repository without entering passwords
  • (From Lee) No Kerberos passwords cached in unencrypted form
  • It should be easy to set up new Hg repositories and restrict push/pull access
  • Each project may need to grant different sets of remote users access to a repository via HTTPS
  • It's acceptable to require extra work to integrate a repository with Trac, but the existence of Trac should not affect setting up non-Trac repositories

Given the above, the solution we arrived at is to:

  • Require local users to access Hg repositories via SSH. This allows the normal Unix filesystem permissions and Mercurial mechanisms to determine read/write access to the repository. The normal passwordless SSH mechanism works fine (from EE machines, I think the SSH servers require passwords for non-EE machines), and there are no issues running the Trac hooks.
  • According to the Mercurial documentation, the allow_push and allow_read sections in a repository's .hg/hgrc file should allow fine-grained control of which users (they still must be authenticated by HTTP server) are allowed read/write access to the repository. If that works as advertised, groups of repositories could share .htpasswd files with the remote user names and (non-Kerberos) passwords. Each repository could explicitly list the user names allowed read or write access, or grant permission to everybody in the .htpasswd -- whichever is easier, likely depending on the number of remote users and how much overlap between projects. It may not work though; Apache ignores at least some .hg/hgrc contents.
    • There's also the option of having a different .htpasswd file for a given repository. The path to the .htpasswd file is specified by the URL matching stanza in the Apache configuration file, so each repository that wants a unique .htpasswd will need its own stanza.
    • Separate stanzas are also required to configure different hooks for the set of repositories served by the URL.
  • For Trac integration, it looks like the REQUEST_URI environment variable is available to the hook script (it's a CGI variable rather than a Mercurial variable) and contains the information needed to allow multiple repositories to share the same hook. It may still be desirable have at least 2 repository collection URLs so that we don't force non-Trac repositories to run a useless hook.
    • Repos under https://lungs/hg/ have no hooks, created in lungs:/projects/mercurial/repos
    • Repos under https://lungs/hg-trac/ have the Trac hook, created in lungs:/projects/trac/repos
  • Passwordless HTTPS access is possible via the [auth] section of users' ~/.hgrc files. Lee requests that all such passwords be different than the users' Kerberos passwords (if they have one) since they are stored unencrypted. All HTTPS users would then need to be entered into the appropriate .htpasswd file. If local users access via SSH and there are only 20 or 30 remote users, this is probably not too burdensome. This might also be the mechanism of choice for projects where access needs to be restricted to a small number of local users -- while it's a slight annoyance that they wouldn't be albe to use their Kerberos password, it shouldn't be too bad since they can put it in their ~/.hgrc file and not have to remember it.
    • It looks to me like allow_push/read only applies to web access, and that the only SSH or file system level access control is via Unix permissions?

So, my proposed course of action is:

  1. Turn off Kerberos password access to the current HTTPS Mercurial & Subversion repositories by setting
    PAM_Enabled off
    AuthBasicAuthoritative on
    
    in lungs:/etc/httpd/vhosts.d/ssl.conf for the Subversion & Mercurial URLs.
  2. Add user names/passwords for any current remote users to the existing .htpasswd file
  3. Add a https://lungs/hg-trac/ URL to the current Apache configuration to serve Trac-enabled Mercurial repositories stored in lungs:/projects/trac/repos
  4. Modify the current hook script to use the REQUEST_URI environment variable to identify the Trac environment
  5. Convert the GMTK CVS to Hg under the Trac-enabled URL.
  6. Ajit can move the MSMS repository under the Trac-enabled URL if he wants Trac integration, or he can request an MSMS-specific URL if he wants to define MSMS-specific hooks or have an MSMS-specific .htpasswd file. His decision may depend on whether Apache respects the allow_push/read. I'll test that soon.

Going forward with some of the use cases Ajit and I discussed:

Large projects like MSMS or GMTK
These will likely want Trac integration. O(10) remote users with non-Kerberos passwords in either a shared .htpasswd file (possibly with per-repository allow_push/read in the .hg/hgrc file) or project-specific .htpasswd files (requiring project-specific URL matching stanzas in the Apache configuration). Projects can be under the https://lungs/hg-trac URL, or a project-specific URL if they want to define their own hooks. Local users access the repository via SSH with group-based read/write permissions.
Small grant writing projects
No Trac integration. These need to restrict access to a small set of local and remote users. Remote users must be entered into a .htpasswd file. Here we hope the allow_push/read really does work over HTTPS, as having to create a new URL matching stanza to point at a project-specific .htpasswd file is probably too much work. Local users can access via SSH if group read/write permission is satisfactory, or else local users can be added with a non-Kerberos password to the .htpasswd file and access via HTTPS. Local (& remote) users can keep their non-Kerberos password in ~/.hgrc so they don't have to remember it. Since this case applies to small groups of users, it doesn't require a large-scale .htpasswd population effort for the entire local user base.
Last edited 4 years ago by rprogers (previous) (diff)

comment:15 Changed 4 years ago by rprogers

Verified that Mercurial access through Apache respects allow_push/read in [web] section of .hg/hgrc file.

comment:16 Changed 4 years ago by rprogers

It looks like a system administrator would be required to change the ownership & permissions of the repository for the case where you want user-level access control.

comment:17 Changed 4 years ago by rprogers

  • Status changed from new to accepted

comment:18 Changed 4 years ago by rprogers

  • Status changed from accepted to closed
  • Resolution set to fixed

This is implemented. See HgProcess and HgHowTo.

Note: See TracTickets for help on using tickets.