NYCGA.net Dev Guide

NYCGA.net Wiki
The New York General Assembly website for coordination and collaboration

The NYCGA.net Website is one of the main focuses of the Technology Operations Group.

The technical instructions below refer to WordPress (the platform used by NYCGA.net) not to MediaWiki (the platform used by this wiki).

Contents

[edit] Server Environments

  • Production: 166.84.136.69
  • Staging: 69.60.9.7
  • Development: 69.60.9.13

[edit] Join the Development Team

[edit] Development Process

A WORK IN PROGRESS!

This will walk you through the process of development and deployment on the nycga.net site.

[edit] Database

[edit] GitHub Setup

  • When starting a new feature, create a feature branch. This should correspond to a "story" in Pivotal Tracker
 git branch feature_name
 git checkout feature_name
  • Keep your feature branch local unless multiple devs are working on the same feature
  • As you work , commit and add as usual. You should be committing several times per day.
  • When you're ready to submit a feature for approval, checkout master and
git merge feature_branch_name
  • then push to origin master
  • Set your Pivotal ticket to "finished" to alert the team the new feature is ready to deploy for testing.
  • To deploy to staging server, pull the master branch to the server. Testers will verify that the feature works, and did not break anything else.
  • To deploy to production, pull the master branch down on top of the production branch on the production server.

[edit] Accessing the Development Server

Commons-emblem-notice.svg Please note: The development environment no longer exists, this information is out of date. 
173.203.98.104        www.nycga.net
173.203.98.104  nycga.net
  • Make sure your DNS cache is clear (OS X: dscacheutil -flushcache, Chrome: chrome://net-internals/#dns "Clear host cache")
  • Goto nycga.net in your browser, you should see the red site header indicating you're looking at Dev)

[edit] Project Tracker Workflow

Commons-emblem-notice.svg Please note: This workflow has changed and needs to be updated.

Bug reports and Feature requests are added to the Pivotal Tracker project for NYCGA.net Each bug or feature is called a story and should have a unique branch made on Github that will be associated with its own story. If a css bug is found on the front page a branch with an appropriate name will be created. Only work related to that bug and story should be done and committed to that branch.

  1. Find a story you can work on
  2. If no one has started work you can begin, click "Start" and then branch from 'master' for that project on github. Call your new branch in the following convention: "20913269_Group_Emails_Broken" where the number at the front is the story ID in Pivotal Tracker .
  3. Write awesome code, get feedback if needed in Pivotal Tracker /IRC/etc.
  4. Commit your changes on your feature branch and send a pull request if necessary). Click finish.
  5. If a story is listed as "Finished" on Pivotal Tracker , that is a sign that system admins should deploy that branch to the staging server for testing.
  6. When the story is live on the staging server to be reviewed, the sysadmin who did the deploy should click "Deliver"
  7. When the Product Owner(s) and QA team finish reviewing your story, they click "Accept" or "Reject". Rejected stories need more work. Accepted stories are ready to be deployed to Production.

[edit] Local Install

To set up a local development instance of nycga.net

[edit] Working with the sanitized DB dump

All accounts have had their passwords changed to "1234". If you need to log in as an admin, but aren't one in production, you can use the credentials "dan/1234".

You can register a user without setting up an SMTP server on your local instance if you look up the activation key in the wp_signups table, and then enter the URL "http://www.nycga.net/activate?key=<key>" in your browser.

[edit] Dan's Ubuntu Guide (tested on 10.04 and 11.04)

Run these commands literally in your terminal one by one, making sure you don't run into errors. Stuff in bold is for humans to read or substitute.

[edit] Apache, mySQL and PHP

  1. sudo apt-get install mysql-server apache2 php5-mysql php5-curl libapache2-mod-php5
  2. sudo a2enmod php5
  3. sudo a2enmod rewrite
  4. cd /var/www
  5. sudo mkdir nycga.net
  6. sudo chown $USER:$USER nycga.net
  7. cd nycga.net
  8. Set up your SSH key in Github (generate a new one using ssh-keygen)

[edit] Installation

  1. If git isn't currently installed, install git: sudo apt-get install git-core
  2. git clone git@github.com:OccupyWallStreet/nycga2.git web
  3. cd web
  4. cp env-sample.php env.php
  5. Put your root mysql database credentials (twice) in env.php
  6. echo create database nycga | mysql -u root -p
  7. cd /tmp
  8. wget http://69.60.9.7/nycga.sql.gz
  9. gunzip nycga.sql.gz
  10. mysql -u root -p nycga < nycga.sql

[edit] Virtual Host

  1. Create /etc/apache2/sites-available/nycga as root and enter the following content: sudo nano /etc/apache2/sites-available/nycga
  2. paste the following into the new nycga file
<VirtualHost *:80>
       DocumentRoot    /var/www/nycga.net/web
       ServerName      www.nycga.net
       ServerAlias     nycga.net *.nycga.net
       DirectoryIndex index.php
       <Directory "/var/www/nycga.net/web">
                Options -Indexes
                AllowOverride All
                Allow from All
       </Directory>
       ErrorLog ${APACHE_LOG_DIR}/error.log
       LogLevel warn
       CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
  1. sudo a2ensite nycga
  2. sudo /etc/init.d/apache2 restart

[edit] Hosts File

  1. On your web client, point your hosts file for nycga.net, www.nycga.net and any subdomains.nycga.net to your ubuntu install's IP. These entries in the /etc/hosts file work for a local install:
127.0.0.1 www.nycga.net
127.0.0.1 nycga.net

[edit] View

  1. Hit www.nycga.net - Good luck!!

[edit] Debugging

  1. For debugging problems, tail your apache logs with tail -f /var/log/apache2/*log and turn up logging by setting WP_DEBUG to true in env.php

[edit] Pea's Guide (tested on Mac OS X)

[edit] Prerequisites

  • Apache Webserver
  • mySQL
  • PHP

[edit] Instructions

  1. Clone the repo ( https://github.com/OccupyWallSteet/nycga2/)
  2. Create a new database, download data and import data (http://69.60.9.7/nycga.sql.gz) into newly created database.
  3. Copy env-sample.php to env.php and replace the DB info to match your local DB set-up
  4. Edit your hosts file (likely at /private/etc/hosts) so that nycga.net and www.nycga.net point to localhost/127.0.0.1 Note: You can use HostWidget or Gas Mask for easy hosts file switching.
127.0.0.1  www.nycga.net
127.0.0.1       nycga.net
  1. If it doesn't exist already, create an events-manager directory in /wp-content/uploads/
  2. Direct your browser to http://www.nycga.net/wp-admin/network (login: superadmin / <pwd>)

Please, report problems to TechOps.

[edit] Jake's Mac OS X Guide (No CLI)

These step-by-step instructions are for those of you who are (like me) more comfortable in a Graphical User Interface than a Command-Line Interface. This was tested in Mac OS X Lion (10.7.2)

[edit] Prep

Before we begin, please download and install the following software:

  1. MAMP. I tested this process using the free edition, version 2.0.5
  2. Sequel Pro. I tested this process using version 0.9.9.1
  3. GitHub for Mac. I tested this process using version 1.1

[edit] Clone Git Repository

  1. Make an account at GitHub.com (you likely had to do this to install the app above).
  2. Visit https://github.com/FLOSolutions/nycga2 and at the top of the page, find the "Clone to Mac" button. This will launch the app and clone the file strucure of the site down to your local computer in a directory of your choosing. I recommend your user's 'Sites' folder (Users/Username/Sites/)
    1. If you are not on the core team and do not have access to push directly to the FLOSolutions repos, you'll instead want to work on your own fork of the repo. In this case, find the 'Fork' button. You will then be asked to select the account you want to fork to and after a few seconds of "hardcore forking action", this should create a "fork" of the repository in your own github account.
    2. Launch the GitHub app on your Mac. In the left sidebar, under GitHub, you should see your own GitHub account there. Click on that and it will show your fork of our repo. If you don't see it, try clicking the refresh button along the bottom of the window.
    3. Once you have found the fork in the app, click "Clone to Computer". It will ask you to select a location to drop the clone. I recommend your user's 'Sites' folder (Users/Username/Sites/) [beware that this function will clone the public repository]
  3. If you browse to that folder in Finder, you should see all the files from the site. Success!

[edit] Get our most recent database dump

  1. Download DB dump from the publicly-accessible link: http://69.60.9.7/nycga.sql.gz
  2. In this version of the database, all private data (email addresses, private messages, etc) have been stripped. All email addresses are reset to "no_reply@nycga.net" and all passwords have been reset to "1234".

[edit] Set up MAMP

  1. Launch MAMP. It will likely ask you to enter your OS X admin credentials. Hopefully it will automatically start your MySQL and Apache Servers. You can tell because there will be handy little green indicators next to both.
  2. In MAMP Preferences:
    1. On the APACHE tab and change the 'Document Root' to the path where you cloned the repo above.
    2. On the PORTS Tab, click "Set to Default Apache and MySQL ports"
    3. Click OK
  3. If you now open a browser and type "localhost" into your address bar, you should get a page asking you to create env.php. This is good! This means you're on the right track!
  4. On the main MAMP window, click "Open Start Page" or just type "http://localhost/MAMP/?language=English" into the address bar of your browser. Here, you will see the default Host, Port, User, and Password of your SQL Database. Note this info for later, or keep this page open for reference.

[edit] Customize your Install

  1. In finder, copy the "env-sample.php" file and rename the copy as "env.php".
  2. Open it in your editor of choice and on lines 7 and 17, change 'root' to your database password. By default, it is already 'root' so there may be nothing to change.

[edit] Import the database

  1. Launch Sequel Pro.
  2. Enter a name ("MAMP SQL"), a host (127.0.0.1), and the username and password from above. Add to Favorites for later and then connect.
  3. Under the "Database" Menu, click "Add Database" and name it "nycga". Make sure that the database dropdown in the top left corner shows that you're now working in your new DB.
  4. Under the "File" Menu, click "Import". Navigate to the nycga.sql file you downloaded earlier. Leave all settings as defaults and click Open. An "Importing SQL" window should pop up with a progress bar. This is a big database, and getting bigger all the time, so this will take a few minutes. This process may throw up some errors. Click "Ignore all Errors". We'll just pretend that never happened, okay? Our little secret...

[edit] Edit your hosts file

  1. Edit your hosts file so that nycga.net and www.nycga.net point to localhost/127.0.0.1
127.0.0.1  www.nycga.net
127.0.0.1       nycga.net
  1. If you have to, reset your cache in your browser. (chrome users: chrome://net-internals/#dns)

[edit] Develop

  1. If you hit www.nycga.net in your browser, you should be rocking and rolling. You should see a big red "DEV ENVIRONMENT" box floating along your right margin.
  2. All you need to do is log in (as any user you want, using "1234" as the password). Hint: If you want superadmin access, feel free to log in as me (jakedeg)
  3. Once logged in, you should be able to access the dashboard (/wp-admin) and the network admin dashboard (/wp-admin/network/)
  4. Play around, edit files on your local install, and commit changes to a new fork. Isn't this fun?!?

[edit] Staggered database importing using BigDump

If you're having issues importing the database using other methods ("MySQL server has gone away"), you might try importing using the BigDump PHP script. Potential settings changes:

  • In BigDump.php
    • $linespersession = 160; // Lines to be executed per one import session
    • $delaypersession = 2000; // You can specify a sleep time in milliseconds after each session
  • If running under MAMP (MAMP/conf/my.cnf (copied from MAMP/Library/support-files/my-huge.cnf)
    • max_allowed_packet = 16M


[edit] Shana's (Occupy Atlanta) Guide for WP+BP site transfer

This procedure was provided by Shana from Occupy Atlanta. We're in the process of verifying that this works, so no guarantees at the moment.

1. Move all current website directories and files to the new site.

2. Create the EXACT same database, user, and password as the current site. OR if you need to change the database info you will need to do that on the wp-config.php and the bb-config.php files located in the main directory.

3. Download a .zip of the entire current database, then Import to the new site database.

4. Make root directory or domain name changes in the database: make these changes in the wp_site and wp_options Table (several places, look closely) You can also run this statement on the wp-options table to change the domain quickly UPDATE wp_options SET option_value = replace(option_value , "olddomain.org", "newdomain.org") it is recommended to double check it afterwards and don't forget to change the Root Directory manually. There could be in a number of places but generally, option_name recently_edited and bb-config-location. Be sure to Make those changes in the bb-config.php and wp-config.php both located in the main directory of wordpress.

5. make sure all folders are set to 775 and files 664

CLEAR UP ERRORS:

If there is a session error: check for a space before an opening php tag in the wp-includes/formatting.php

If there is a permalinks issue:

Make sure your .htaccess file is writable.

Now, try switching the permalinks option back to manual, save changes, clear the cache and see if it works. If it does, switch back to the original option, save changes, clear the cache and see if it works. If it does not, Delete your .htaccess file, then add a blank one, and repeat earlier steps. If that does not work, contact your Server Admin and ask for them to check on the apache settings.

IF THERE IS A DOMAIN NAME CHANGE: There are several scripts you have to run on the data base to keep from having all of the links from old BP posts pointing to the original domain, but those links don't affect the install. Here is one of the statements:

UPDATE wp_bp_activity SET action = replace(action , "olddomain.org", "newdomain.org")

Go through your database and look for links that include the old domain. Replace the current table name wp_bp_activity with new table name that need the domain changed and replace action with the field that contains the link.

[edit] Aeon's FreeBSD notes

If you're experiencing mysql import error:

ERROR 2006 (HY000) at line 4459: MySQL server has gone away
  1. FreeBSD very commonly runs mysql-server without a configuration file. This is very normal so dont worry if you dont see one. There are several example files for this located in: /usr/local/share/mysql. I chose to use the my-medium.cnf, you may choose any of the example .cnf files listed there. copy the file you choose to /usr/local/etc and name it my.cnf.

example: cp /usr/local/share/mysql/my-medium.cnf /usr/local/etc/my.cnf

  1. Now edit /usr/local/etc/my.cnf and look for: max_allowed_packet = 1M Change this to: max_allowed_packet = 16M
  2. Restart your mysql-server and you can now import this file without any errors or issues.

Editing your hosts file /etc/hosts

To point nycga.net to localhost the proper format is:
127.0.0.1               localhost nycga.net www.nycga.net

If you experience the following, your hosts file is not correct:

Warning
http://www.nycga.net/
A page on the public Internet requests data from your computer. 
For security reasons, automatic access is blocked, but you may choose to continue.

[edit] Back-end/Server Documentation

[edit] Server Environment

  • Development server - 173.203.98.104
  • Staging server - 69.60.9.7
  • Production server

[edit] Backup Strategy

  • Hourly mysqldump of the live database, rsync’d to the staging server (i’d like to do something more realtime, like rsync the db directory offsite)
  • Hourly rsync of user-uploaded content to the staging server (again, since it’s rsync, this can be done more often)
  • Code is managed in git, so the production branch exists in several places (Github, staging server, developers’ local machines).
  • Periodic rsync and tar the entire site somewhere safe every few minutes, for disaster recovery
  • /etc/ snapshots are stored in a git repo (http://www.jukie.net/~bart/blog/20070312134706)

[edit] Installing for your GA

it’s important to determine if you need multisite or not. If working groups will not want their own websites, you may not need it. That being said, it is relatively simple to add that option even if you don’t you it immediately. The information here will include the MS setup.

1. Install WP on your servers (one for the staging server and one for the live site)

2. Go to wp-config.php.

  • above this line:
/* That’s all, stop editing! Happy blogging. */:
  • insert this line:
define(‘WP_ALLOW_MULTISITE’, true);

3. Log into WP Dashboard and go to Administration > Tools > Network Setup to see the screen where you will configure certain aspects of your network.

4. Choose you preference between sub-domain and sub-directory and go to next screen

5. Add the code that appears to the wp-config.php and .htaccess files to complete multiste setup

6. Logout and log back in to WP Dashboard (Full WP Networks instructions here: http://codex.wordpress.org/Create_A_Network)

7. Download plugins and themes from https://github.com/OccupyWallStreet/nycga2

8. Upload these files to wp-content directory

9. When you log back into WP Dashboard on the top right where it says Howdy, Username, click on the drop down menu and you’ll see an option for Network Admin, go there to activate plugins and themes, use this guide as reference: http://wiki.occupy.net/wiki/NYCGA.net_Dev_Guide Note: Some BuddyPress plugins will only be available on the Network Admin Plugins for activations. DO NOT network activate the Events plugin.

10. Configure BuddyPress settings on Network Admin Panel, this should only need to be done once. After that everything should be able to be managed from the front-end of the website based on your userlevel access.

In regards to you own GitHub account, you can simply clone what we have for your needs.

I think it is free to use Pivital tracker, however, we have recently deployed and open source software that has many of the same features, feel free to create an account and use for website project management as well. You can do so here: http://collaborate.occupy.net.

[edit] Plugins

[edit] Active Plugin List

Commons-emblem-issue.svg THIS LIST IS NOT UP TO DATE. WE NEED A WAY FOR IT TO BE GENERATED LIVE OR SOMETHING

Below is a list of all the Wordpress plugins we are using on the site. Some are right out of the box, some have been modified, and some have been built by us from scratch. We will try to break each down so you understand its purpose and its origin.

Commons-emblem-notice.svg Please note: If you are NOT using WordPress Multi Site, then install both, Network Active and Active plugins.


Plugin Network
Active		 NYCGA.net
Active		 Purpose History
Adminer Y
Advanced Most Recent Posts Mod Y
Akismet Y
BP-Registration-Options
BP Groupblog Y
BP Group Files
BP Group Organizer Y
BuddyPress Y
BuddyPress Activity Plus Y
BuddyPress Custom Posts
BuddyPress Docs Y
BuddyPress Group Email Subscription
BuddyPress Groups Extras
BuddyPress Group Tags Y
BuddyPress Like Y
BuddyPress Rate Forum Posts Y
Capability Manager
CD BuddyPress Avatar Bubble
Configure SMTP Y
DirtySuds - Embed PDF Y
Dynamic Content Gallery
Dynamic Content Widget
Enhanded Text Widget
Events Manager Y
Facebook Button Plugin
Facebook Like
GA Contact Tab Y
GA Status Tab Y
GA Group Categories Y
GoGadget Weather and Weatherforecast Widget Y
Google Analyticator Y
Hikari Email & URL Obfuscator Y
IntenseDebate
Invite Anyone
LiveStream.com Thumbnail Y
Multisite User Management Y
OpenID Y
Q2W3 Post Order Y
Really simple Facebook Twitter share buttons Y
ShareThis
SI CAPTCHA Anti-Spam
Simple:Press
Site UnderConstruction
Tweet Blender
Unconfirmed Y
Viper's Video Quicktags Y
W3 Total Cache Y
WangGuard Y
WordPress HTTPS Y
WordPress Importer Y
WP Minify
WP Security Scan Y
WP Super Cache
WPtouch
XRDS-Simple Y
Zoom.it Y

[edit] Available Shortcodes

First of all, please not that you don't need any shortcodes at all to embed media from the most popular sources. For more info on this, check out http://codex.wordpress.org/Embeds.

If you need to do something else, you can use the shortcodes provided by the following plugins:

[edit] zoom.it Wordpress

  • Zoom.it: [zoomit id, width, height]: "id" is the identifier for an image uploaded for processing to the zoom.it site; "width" is the desired width of the blog image, or "auto"; "height" is the desired width of the blog image, or "auto"

[edit] TGN Embed Everything

  • IFrame: [iframe tgn.tv 604 500]
  • YouTube: http://www.youtube.com/watch?v=WDLgEyJ3SlE
  • PDF (without Flash!): [pdf tgnbooks.com/the-great-gatsby.pdf 604 700]
  • PowerPoint (without Flash!):[powerpoint tgnbooks.com/best-caricatures.ppt 604 700]
  • Google spreadsheet: [spreadsheet 0AtFXq8XyejTEdGNEZm1NZUJ3V3QwXzIwZnQ1ckh0MlE 604 300]
    • Show any sheet: [spreadsheet 0AtFXq8XyejTEdGNEZm1NZUJ3V3QwXzIwZnQ1ckh0MlE 604 300 sheet=2]
    • NOTE: You must "Publish" the spreadsheet in Google Docs by clicking "Share > Publish as a web page" to see it in WordPress!
  • Google document: [document 1t0C9HghMO4ttIKbUYxVC2sn5nXi9hXldVS1okjWzTP4 604 300]
  • Vimeo: [vimeo http://vimeo.com/2481023]
    • Remove the space between "http://" and "vimeo.com" (Purposely added to prevent rendering the video on wordpress.org!)
  • Metacafe: [metacafe http://www.metacafe.com/watch/1203580 ]
  • Dailymotion: [dailymotion http://www.dailymotion.com/video/xg7km9 ]
  • VideoReadr: [videoreadr Uz7fOLDr2JM nmrdsxtb 604 370]
  • TIFF: [tiff tgnbooks.com/electronic-circuit.tif 604 840]

[edit] Viper Video (if you activate it for your blog)

[edit] DirtySuds PDF Embed

[edit] User Meta Shortcodes

USAGE:

use [userinfo field="fieldname"]some content[/userinfo] or [authorinfo field="fieldname"]some content[/authorinfo] shortcodes in your post content to show the "fieldname" meta value from the user-meta for the current logged-in user or for the post author without editing your theme files.

EXAMPLES:

[userinfo field="last_name"]Template:Empty[/userinfo]

returns the last name of the current logged-in user. If no user is logged in then the value is empty string

[userinfo field="user_login" if="admin"]You are the admin[/userinfo]

this shortcode tests the field against the value and if true it displays the included content

[userinfo field="user_login"] is your username and you are reading a post by [authorinfo field="user_login"]Template:Empty[/authorinfo].[/userinfo]

this will display the username followed with the processed content. Note: never use the nested shortcode same as the enclosing shortcode. This will produce unexpected results!

List of some of the available meta field names: ID, user_login, user_pass, user_nicename, user_email, user_url, user_registered, display_name, first_name, last_name,nickname, description, user_level, admin_color (Theme of your admin page. Default is fresh.), closedpostboxes_page, nickname, primary_blog, rich_editing, source_domain

[authorinfo field="last_name"]

returns the last name of the current post/page author.

[authorinfo field="ID"]

returns the user ID of the current post/page author.

List of some of the available meta field names: user_login, user_pass, user_nicename, user_email, user_url, user_registered, user_activation_key, user_status, display_name,nickname, first_name, last_name, description, jabber, aim, yim, user_level, user_firstname, user_lastname, user_description, rich_editing, comment_shortcuts, admin_color,plugins_per_page, plugins_last_view, ID

If you add custom user meta via additional plugins, then the meta should be available for the above shortcodes

[edit] Audio Player

[audio:http://www.yourdomain.com/path/to/your_mp3_file.mp3]

[edit] Policies for NYC GA Digital Properties

Admin use of power log - Record of major administrative actions

[edit] NYCGA.net Privacy Policy

[edit] Best Practices

[edit] Security

The staging server is/will be accessible via SSH2 with key authentication only.
 
Brought to you by OCCUPY TECH