You may have noticed...
There's a new tab at the top of my site. A great big shiny new button that says Syntax Compare. That's right! The "Dynamic, Programming Language Syntax Comparison" page is up and running!
A lot of work got done this past week, and there are quite a few experiences I ran into I'd like to share now that the project has been released. Of course just because the page has been launched, doesn't mean its finished. There is always just one more functionality enhancement; a practically endless amount of data entry potential; and I've already had a clever idea for a completely new content section for the page!
But before we get ahead of ourselves, lets catch up with where we left off last time.
Cleaning up after myself...
One of the first things I jumped into after the last post was going back and normalizing the database. The issue was that the database had been using the syntax 'content topic names' as Primary keys in a few tables. Names like "Data Types" and "Constructors & Destructors" which are used to display how sections of syntax are grouped together. So if a situation ever arose where one of those sections needed to be renamed, there were multiple places in the database where the name would have to be updated.
I had been vaguely aware of this issue before, but at the time it did not seem likely that those names would need to be change. Now that the project was getting to the point where a large amount of data was going to be added to the database, it seemed like a good idea to fix this problem now rather then let it crop up later when the data would be less agile.
So the first step in normalizing the database involved restructuring my tables so they had new numeric unique ID's that would never be modified. This change needed to be applied everywhere those names had been used as a primary or foreign key though. Since there was already some syntax content in the Database already, I had to write a small SQL script to ensure the new PK id's matched up with the old values. It was only AFTER those new columns had been added and populated that the actual table structure could be altered to define the new PK and FK's. Re-designing the database to be better organized like this makes the data more robust in the long term, and helps save a lot of space.
I was worried that this would result in some fairly ugly and monolithic multi join SQL statements, but after a few changes it turned out to not be that big of a deal. Most of the java script functions managed to survive the change with barely any updates as well. Overall the database restructuring ended up being a success. The lesson here is that its important to take the time to set the foundation right. Luckily I caught it early enough that it wasn't to hard to deal with.
Continuing my theme of cleaning things up, I went back to 'normalizing' my css as well. This meant striping out all of the temporary css that had been used to make the site bearably readable while testing, and placing it all in a single css file. There were a few points during this process when pages content started jumping around and losing all of its formatting, but getting this sorted now would make site beautification much easier later on.
While I am not a CSS guru when it comes to webdesign, one of the lessons I had taken to heart was that fairly subtle changes can make a huge difference to aesthetics. Something had to be done with the navigation and interface of my site, and the temporary header from the CS50 class needed to go as well. So rather then trying to come up with a huge expansive layout and look for the site, I focused on smaller individual changes to polish up the design.
Simple things like adding rounded edges to link buttons, using very subtle shadowing, and creating distinct but subdued hover over effects for buttons go a long way to make the site more appealing. I immediately discarded the idea of using a dark color pallet, to sticking with the light grey/blue combination so as not to distract attention away from the main comparison table. The rest of the css work was focused on making sure all of the site elements were positioned correctly.
There were a few more complex site design changes I didn't get to implementing. Things like getting the 'page on top of a static background' look similar to this word press theme. But these type of changes were proving to be a time sink to try and teach myself. For the most part, I didn't want to make the design too involved either since the primary focus of the site is on the syntax comparison table. The main issue though was that I still had a huge time sink task ahead of me that was more necessary for the completion of the site...
Data Entry, the home game
One of my goals for releasing this site was to add at least two more languages in addition to the three from the Harding site. Now, I could play you a data entry montage of how I flushed out the syntax content for PHP and Python; but I would rather not relive the memory if I can help it. Even with the data entry tools I created for myself, it proved to be a very time intensive task.
Plugging in the C#, VB.NET, and Java languages was a straight forward but time consuming task. All of the content had already been written and was ripe for copy-pasta, but it still had to be formatted correctly by adding the line breaks, formatting the comments, and checking the spacing. This had to be repeated for over a hundred separate syntax categories for each language and took the good part of a day for each of those languages.
On the other hand, the PHP and Python syntax content was created entirely from scratch. Each of these languages took about two days to fully populate in the database. Part of this was because I was trying to teach myself the languages during the process. While I have programed with both PHP and Python before; it had been a while in the case of python, and there were certain more in depth properties of both languages that I had never needed to use before. The entire process was a great way to quickly learn a language and find out about some of the more obscure functionality within them; but it was an extremely time consuming task.
What I learned about learning...
One particularly striking difference between those two languages was that the PHP documentation was really outstanding. The documentation was coherently organized and explained itself well, it was fairly easy to search and find the term you were looking for, and each page was accompanied by a bunch of user or community post that helped shed light or go into more depth on particular topics.
By contrast I found the python documentation rather lacking in comparison. Maybe this is partly because of the incompatibility of python versions 2 and 3, and so they are trying to maintain documentation for multiple versions of Python at once. But searching for specific terms and contents was a lot less succesfull in returning information relevant to what I was looking for. There were several times when the documentation would take for granted what it considered to be 'common knowledge', or where it would skip explaining intermediary steps between concepts. I wound up having to read through several sources before I figured out how to connect the dots between the missing steps.
At one point I had also intended to try and add Haskell syntax as well. It is a drastically differently structured language compared to some of the more 'common' programming languages people may be used to. After spending a few hours though trying to figure out how to simply integrate command line args ( "cause we are all ':t (IO string)' which won't play nice with normal ':t (strings)', nyah") into a program though, I gave it up as not having a good time to 'worth it' ratio right now. It's something I still want to look into sometime, but now is not that time. On the other hand it does look like there is a large amount of useful Haskell documentation out there.
Pushing the content live
Skipping ahead past some more site revisions and polishings, the last major issues I ran into was trying to push the web project files to my domain. While my development machine had been testing the site at the root level of its 'localhost domain', my web hosting already has word press installed at the root. In addition to that, I wanted to be conscious about having some of my back end and supporting php files properly secure. This meant having to change the directory structure and path relations in all my files.
Eventually I found out I needed to create a second ftp account (as appose to the default one my web hosting provides) in order to access the below root directory and store files that only my own scripts can access. This meant I could now separate my supporting libraries and view pages, from the controller php files that users will call. But both of these locations required different path names which no longer played nice with relative paths.
Using proper form, I had already defined path constants in a config file which the php files reference. The only trick was; since that config file would be separated from the controller files now, I created a new config file relative to the controllers that would point all of them to the new location of the original config file. If any future changes require altering my directory structure again, I should only have to update the path names in a single place.
White screen of limbo
After finally pushing all of the files to the domain, I entered in the new URL annnnnd.... white screen. No feedback, no error response. Nothing. Something had gone horribly horribly... no where.
In fact this turned out to be a fairly obscure bug to solve. Luckily I was saved by a forum post on the edX forums. It seems there was a certain PHP syntax, which involved using square brackets to group together a dictionary of values to pass in a render function, which was only available on the most recent version of PHP (5.4). If a web domain did not support this version of PHP, all of those calls in several of my files would have to be changed.
Fortunately, while my web domain defaults to PHP 5.3 they give users the option to use PHP 5.4 if they are willing to deal with the most recent version / 'early adapter' quirks.
The tech support at my web domain had actually been really helpful, as I had been in contact with them while trying to deploy my site. Mostly I was just communicating to them for peace of mind and to make sure I didn't accidentally make any of my php setting files public. But they were able to give me a lot of directly useful information, and refrained from simply telling me to look up their online faqs which I had already perused. In that regard I'm willing to recommend my web host provider if anyone is looking for cheap domain hosting.
Passing the Milestone
The only thing left in this chapter is to submit my project to the CS50 project expo. The next major milestone for this project would be getting the data entry web tools polished to a point, where I could start letting outside people update and add new syntax content. But that presents a number of new problems, such as putting in a robust audit system to track changes, and dealing with user validation in a way that keeps out spam bots or malicious defacers.
There is of course an endless amount of tweaks, data entry, spelling corrections, and new content potential for this project; but its about time to start shifting my focus to new endeavors. There are a few things that have been neglected with this blog I would like to address. Then, my next major project will be focused on creating a Khan Academy exercise module.
Until then, thanks for following along with me on this project, If you've been reading this long! I hope you find the Dynamic Programming Syntax Comparison page useful.