Installing on Leopard

I'm moving my hosting from NESCent's servers to a server in my lab (running Mac OS 10.5). Having immediate access to the database structure will be helpful in continuing TreeTapper's development. Installing and configuring postgres on the Mac has taken some work. One useful site so far is here (though note that the latest version of postgres is 8.4, not 8.3). I'll post more as progress is made...

Viewing missing methods/software, part VI: Processing done!

I've written the code to generate the missing methods/software tree diagram using Processing. The relevant page is http://treetapper.nescent.org/findneed.php . I've designed it so that as data streams back from the database via a PHP script, Processing draws this on the diagram in real time. As users update the sortable list of options (tree type, general question, etc.) (using YUI drag and drop), a Javascript function updates the string (stored in a Javascript variable) that is passed to the PHP script. Processing checks this string (so, Java talking to a Javascript object), and if it has changed, Processing closes its old connection to the PHP script and opens a new one using the new options. It thus dynamically updates the chart and is far faster than Google Maps API. It's also easy to do sophisticated animations easily in Processing: currently, I have the nodes flying out of their parents, zooming and shifting the image, and point highlighting on mouseover. These actually don't slow down the rendering: I have the script written so that it only adds a new node to the diagram once the previous node has reached its destination, which sets an upper limit on rendering speed, but I override this and put many nodes on the tree at once if there's a backlog (>10 nodes read from the server but not drawn yet). There is rarely such a backlog, indicating that nodes are being drawn as fast as the server is passing them to Processing. It's pretty cool to be able to visualize where our field needs work (or, rather, where I need to fill in the database) using a dynamic interface, and also surprising that it wasn't too bad to program (especially considering I didn't know Processing/Java, Javascript, PHP, or Postgres when I started this in November 2007). Here is a video showing the new site being used (also available here); you may need to widen your browser window to see it all. As always, suggestions are encouraged.

Updated schema

I've updated the schema slightly, keeping tracks of citations in different tables rather than just keeping one citation field in the reference table. This way, the number of citations over time can be stored (so the rate of increase of citations can be recovered, as well as the number at any point in time). For right now, "citations" will be the number of hits in Yahoo for the title, year, and lead author last name of a reference, since ISI and CrossRef both require payment to use for citation info and Google Scholar's terms of use prohibit display of info from that site (see Rod Page's post about this).

Database schema

Here is the schema for the TreeTapper database. Entries are only shown after being verified by a DB curator (the approved field in each table): this (I hope) will allow users to add missing terms while allowing me to keep the list of terms from getting too disorganized.

Progress so far

TreeTapper is hosted by NESCent and uses a PostgreSQL database as a backend, an assortment of PHP scripts for generating pages, and the Yahoo User Interface Library (YUI) for a front end. Reasons for these decisions:

• NESCent hosting: Free, stable, well-supported, appropriate for the project, and allows things like use of mod_rewrite to make creating RESTful site easier.
  
• Postgres: NESCent supports this rather than MySQL; triggers and views will be useful (though now also allowed in MySQL). I had thought about using arrays for some fields (a Postgres-only feature), such as when a particular bit of software can read multiple tree formats, but will instead use additional tables, following recommendations from NESCent's IT staff.
• PHP scripts: I had looked into using Ruby on Rails, CakePHP, or other frameworks for development, but there seemed to be a lot of overhead in learning them for the benefit I'd receive.
  
• YUI: This is one of the many libraries for Ajax development. YUI is feature-rich and has both great documentation (with many examples) and an active user forum, both important for me.

I've also created the database schema and a way to securely log in (using PHP sessions). Next, I'm adding the ability to store and look up references in Connotea (using a Perl API) and other reference-parsing tools so that I can start adding papers to the database; I'll also create forms for adding methods and software and start actually adding information.