How to set up a development instance

Architecture

There are four possible instances in the FSP development environment:

1. dev.freestateproject.org
2. dev1.freestateproject.org
3. dev2.freestateproject.org
4. dev3.freestateproject.org

Each of these points to a folder link (note: not a folder, but a link that can point to folders) in the /home/freestat/public_html directory on the development server, and this folder link is named after the development instance (i.e. dev, dev1, ....). Each of these instances is also associated with a unique database named after the instance (freestat_dev, freestat_dev1, ...), and a unique user with the same name as the database.

An anonimized copy of the production database is copied to the development environment once a week, to /home/freestat/anondb/fsp_anon.sql.gz. This copy can also be generated at will when required. On the development server, an hourly cron batch processes any fresh copies of this file into four files ready to be loaded into any of the development databases (named dev.sql, dev1.sql, etc.).

Procedure

To bring up a new instance of a development environment, follow these steps:

1. Pick one of the four development environments. Check with the FSP IT mailing list if anyone else is using it. For the following steps we assume that the dev1 environment was chosen.
2. Get the database password from the current settings for the selected development environment (e.g. in dev1/sites/dev1.freestateproject.org/settings.php).
3. Reload the database from the latest anonymized production database copy (note that the password will be requested):
   

   cat ~/anondb/dev1.sql | mysql -u freestat_dev1 freestat_dev1 -p
   

4. Export from the source repository the branch of code that will be used in this environment into the public_html folder. E.g. if the trunk will be used (see the source control topic for the value of $FSPSVN):
   

   cd ~/public_html
   svn export $FSPSVN/trunk

5. There will be a folder named "~/public_html/trunk". Rename that to something unique, in order that it will not be overwritten by another developer:
   

   mv trunk dev_stuff

6. Set the database password in the new copy of the code to the correct password for the dev environment (in ~/public_html/dev_stuff/sites/dev1.freestateproject.org/settings.php).
7. Note where the current folder link points to:
   

   ls -l ~/public_html/dev1
   

8. Move the appropriate folder link to point to the new code:
   

   rm ~/public_html/dev1
   ln -s ~/public_html/dev_stuff ~/public_html/dev1

9. Remove the folder that the folder link pointed to previously (e.g. old_dev1), unless it needs to be kept around for some purpose:
   

   rm -rf ~/public_html/old_dev1

Source control

The FSP Drupal site code is under version control. All items contributing to site functionality, including Drupal core, all modules, and all themes, but excluding the files subdirectories, and settings.php files are kept in the source repository.

Git & GitHub

The source repository is managed by Git. See the Git User Manual, or this Tutorial for how to use Git. All code is kept in a central repository at GitHub.

Repository Management

The source repository is managed very simply:

1. All code is kept in a central repository, and changes pushed back to this repository before staging / production releases.
2. Developers clone the central repository on their development systems (either personal or the shared FSP development systems).
3. Changes are made and tested on the development systems, on the master branch.
4. When changes are ready to be released, they are pushed back to the remote master branch.

Releases

Releases are made from tags on the master branch. See the release process for how releases are done.

How to get started

1. Create an account at github, and add your public SSH key to the account
2. Post your request for access, with your github account to the FSP IT mailing group
3. You'll be added to the FSP-Website repository on GitHub
4. Clone the repository
5. Code away!
6. Push your changes to the main repository when they're tested and ready to go
7. Either release the code, if you have access, or request a release be made on the FSP IT mailing group

Release Management

The production website has a clone of the central FSP website git repository in /var/local/websitesrc. This repository is updated with the remote central repository during a release, and then a release is generated using git archive.

Release numbering

Major releases, containing new features, are sequentially numbered. The release branch and directory name on the production server are of the form:

release-XXX

with XXX the release number.

Minor releases, containing hotfixes for bugs, etc., are numbered as extensions of the last major release:

release-XXX.Y

with Y the sequential minor release number, starting at 1.

Release script

The release is executed using a script, /usr/local/bin/wsrelease, with a parameter indicating the release tag name (e.g. release_6-21). The release destination is /var/www/release_X-YY.

The script does the following:

1. Takes both sites (FSP & Porcfest) offline
2. Fetches the master branch from the repository to a local repository
3. Exports (via git archive) the release tag specified on the command line
4. Extracts the export into a directory named after the tag
5. Relinks the website folder to the new release
6. Brings the sites up again, with drush

See the release script for more information.

Preparation

1. Document the new release at the top of the release notes in /dev/release-notes.txt, incrementing the release number appropriately.
2. Push all local changes on the development system (master branch) to the central repository: git push
3. Tag the release, with the next release number: git tag release_X-Y
4. Push the tag to the central repository: git push origin --tag

Release

1. SSH into the web server
2. As root, run: wsrelease <release-tag> (e.g. sudo wsrelease release_6-21)

Roll-back

If an error occurred during rollout, roll back to the previous release, and clean out the partial release:

1. If needed, link /var/www/www.freestateproject.org back to the previous release, e.g:
     cd /var/www; rm www.freestateproject.org; ln -s release_6-20 www.freestateproject.org
2. Remove the partial release directory: rm -rf /var/www/release_6-21

Note that the release script will throw an error if a directory with the same release tag is found.

Post-release

Send a notice of the release to the fsp-it group, with a link to the release notes.

Registration Management

This documents the current registration process, the transfer of registrations between the original system, and the history of registrations. It is a work in progress.

User Types

1. Member: A person that registers with the FSP as either a Participant, Pioneer, or Friend
2. Participant: A person not residing in NH, that undertakes to move to New Hampshire.
3. Pioneer: A person wishing to support the FSP, but already residing in New Hampshire.
4. Friend: A person that does not intend to move to New Hampshire, but is supportive of, or interested in, the movement.
5. User: Anyone that has a user account in Drupal. Not all users may have registered as a member of the FSP.

Statistics

Already in New Hampshire

This count is shown below the participant count in the site header. The count should consist of the following:

1. 256 pre-vote residents of New Hampshire (per Seth Cohn).
2. All participants that moved to New Hampshire after registration.
   Effectively, the following states are considered 'moved to New Hampshire after registration':
   1. Anyone that has a type of "Participant", but a resident state of "New Hampshire"
   2. Has a value of "Already Moved" in their move date registration field as part of this group.

Roadmap and Status

Find attached a spreadsheet showing the development roadmap and status. The IT Plan for 2006 is also attached for reference.

Mission

The FSP Information Technology Department (FSP IT) supports the FSP mission by providing IT services to all FSP stakeholders as represented by the various FSP departments. These IT services are primarily content and functionality that are found on the FSP web site. Every web site function shall have an owner among the FSP leadership, and every FSP leader should have a clearly defined area on the web site which he maintains and improves. As the FSP is an activist organization facing significant challenges and competition, FSP IT services should reflect best practices among the FSP's peers: namely, charitable, libertarian and activist organizations.

Goals

As a virtual, decentralized, web-based volunteer organization that owes its conception, existence, and success to the Web, the FSP needs to make maximum use of web technologies. Googling the buzzword "Web 2.0" will provide a wellspring of inspiration. Throughout this document, functionality and goals are divided into three categories: Content, Community, and Collaboration.

Primary goals are:

1. Consolidate applications and servers into one Content Management System (CMS) on one server in order to facilitate administration, share data, and provide a coherent platform on which to build future functionality.
2. Reorganize content according to the Center concept, with specific Departments (and individuals) responsible for each Center. Shift the emphasis from dry information towards persuasion, especially promoting New Hampshire and community.
3. Build community with social networking features, including rich profiles, buddy lists, organic groups, and karma-based reputation tracking.
4. Support decentralized project collaboration among the FSP community members (Participants and Friends), in order to make better use of our greatest resource.

Attachment	Size
FspItPlanAndStatus_050708.xls	34 KB

Accounts and Resources

Following are all the accounts and resources used by the FSP IT infrastructure. All account passwords are centrally stored at the Clipperz site, and administrators have the master password to this account to recover individual username / password pairs for other accounts.

Provider	Description	Account ID / Username
Clipperz	Password manager, stores all account passwords used by the FSP IT team / systems.	fspadmin
Google Apps	Hosts email for freestateproject.org domain. No central account admin.	freestateproject.org
Google Analytics	Google Analytics account	UA-1496848-1
EasyDNS	Hosts freestateproject.org DNS service	adamrick
John Companies	Web site hosting company