Markup normalization

A small selection of the Vim statements required to normalize every possible variant of shitty markup entered by copy and paste online editors into 35,000 articles over the last eight years:

:%s/,|||,/=nr2char(11)/g
:%s/,|||/=nr2char(11)/g
:%s/""/=nr2char(21)/g
:%s/"//g
:exe '%s/' . nr2char(11) . '/","/g'
:exe '%s/' . nr2char(21) . '/"/g'
:exe '%s/"$//g' # add a
:%s/^/"/g
:%s/<br /><p><br />/</p><p>/g
:%s/<p></p><p>/<p>/g
:%s/<BR><br />/<br />/g
:%s/<br /><p></p><p>/<p>/g
:%s/<p><br />/<p>/g
:%s/</p></p><p>/</p><p>/g
:%s/</p><br /><p>/</p><p>/g
:%s/<br /><p>/</p><p>/g
:%s/<p><p>/<p>/g
:%s/<P>/</p><p>/g
:%s/</p></p>/</p>/g
:%s/<br /></p><p>/</p><p>/g

Two more things: 1) Anyone who’s ever tried to tell you to use find and replace in bbEdit for large files is dead wrong. 2) College Publisher, you suck ****. ‘,|||,’ is not a valid delimiting character. Quit being malicious.

Lastly, if I’ve thought ahead, I would’ve tracked invalid markup against prevalence and date range. That would’ve made for a fascinating anthropological study.

Teasing WordPress posts with YQL, jQuery, JSONP and iframes

Teasing WordPress posts with YQL, jQuery, JSONP and iframes. In Ivar Vong’s first post for the Daily Emerald web development blog, he discusses an ingenious workaround for bringing outside content to a College Publisher homepage. Using YQL for external data could be an intriguing user experience experiment. The CUNY J-School homepage currently uses Simple Pie to pull at least six RSS feed and, when the cache is being refreshed, the page load time can spike.

Campus directories done right

Not to throw too many tomatoes, but the Daily Emerald made a very “newspaper” mistake today with their website. I’d like start a discussion about “the better way to do it.”

Case in point: The Daily Emerald, I believe as a part of their magazine edition for IntroDUCKtion, created a campus directory. The directory includes dozens upon dozens of email addresses, URLs, and phone numbers for student organizations and sports at the University of Oregon. In the print magazine, which I don’t have access to because I’m in Portland, I’m sure this list of contact information is beautifully presented in an approachable, useful format. Unfortunately, this same list made its way into the website as a long, ugly, flat text file:

Daily Emerald Campus Directory - July 13, 2009

In my humble opinion, there’s a lot of room for improvement.

What if, instead, we approached this directory as the database that it really should be? This web-native directory would have profiles for every student organization much like students can have profiles on Facebook. I’d be able to search for organizations based on the name, the location on campus, people currently involved, the mission of the organization, tags, etc. If I found a organization I was interested in, I’d click through to their profile. The profile would then give me access to all of the contact information I might need in addition to the most recent or popular articles, images, videos, updates from the campus’ microblog, etc. There’d be a small wiki section for the organization or sport where I could read up on its history and know that the information I was getting was true because it had been curated by the beat reporter.

I see at least two advantages to this approach, in addition to making all of the information much more accessible (versus the flat text file). One, you’d only have to build this once. Two, you’d save the reporter or designer a lot of time having to search for the most up to date contact information because they could just pull the information from the database as they’re creating the print product.

Think of role of the student news organization less as a newspaper and more as a platform for impartial, accurate community information to be shared.

Daily Emerald on strike, and the evolution of the newspaper

Around 8:30 this morning, Kai Davis (or @ninjakai) twittered something about the Oregon Daily Emerald being on strike. The initial image in my mind was one of people picketing in the street, and I couldn’t honestly guess as to what they would be striking about.

Then I read the editorial.

Continue reading “Daily Emerald on strike, and the evolution of the newspaper”

Free strategic advice for the @dailyemerald

Last night, I realized we’ve started bitching about the Daily Emerald in the peanut gallery without offering any positive advice for change. I’d like to offer my thoughts on how to turn the struggling newspaper into a successful digital news enterprise.

Step one: hold a transparent weekend (or weeklong) jam session to develop a strategic plan. Invite as many intelligent stakeholders as you can to a retreat, and put together a website for that retreat with the agenda, list of everyone involved, and goals. It might also be useful to have a open community forum in the week preceding to hear strengths and weaknesses from the perspective of the audience, or launch a website where the community and submit and vote on ideas for the news organization. When retreat happens, however, make it open and participatory. Make sure everyone at the retreat is documenting the discussion on Twitter, and livestream as much of the discussion as you can. Have a designated “community manager” for the retreat who looks for suggestions from watchers and brings those to the meeting. Tap the intelligence of the digital crowd, especially because you’ll be able to bring even more smart brains from afar.

Step two: campaign over summer 2009 amongst the Daily Emerald alums to raise the funds necessary to implement the strategic plan. Shop the plan out to them to get their feedback and insights, and use CRM (or customer relationship management) software to track these interactions. When I left, they were using a FileMaker database system and analog mail. I would ditch this system immediately, and my first investment would be software like Salesforce.com (which a news organization could also use to sell advertising more effectively). Using the new CRM, it would be wise to fundraise amongst the alums who want to see their old newspaper experiment with this platform called the internet. Including them in the process, by sending them the strategic plan and a link to the website with an archive of all the video, will make them more invested in the process (if they like what you’re doing at least).

Step three: implement the strategic plan starting in Fall 2009. If I were the publisher of the Daily Emerald, these three are of many things I would attempt to drastically right the direction of the news organization:

  • Quit the College Publisher habit. Being on a locked, proprietary content management system is probably the worst foundation you could have for a digital news organization. Focus heavily on recruiting a few developers out of the computer science program, and build a basic website on Django that you can grow from. If you ask nicely, the Daily Gazette at Swarthmore or Daily UW might be willing to lend enough code to get you started.
  • Move to once a week in print. I know that this would be very, very difficult, especially because the bulk of revenue comes from the print product, but it needs to happen nevertheless. Necessity is the mother of invention. Do it, and publish daily online.
  • Empower your community. Break down the ivory tower, and hold workshops to teach interested community members how to report on the issues they’re passionate about. I am quite certain that club sports at the University of Oregon don’t get the coverage they deserves, and there are probably at least several people who could tweet at games and submit high quality images for a photo gallery.

Right now at the Daily Emerald, though, they’re going about it the API emergency meeting way, and this is just one of the many reasons I think startups make more sense in this climate. I mean, look at all of the effort it’s going to take to turn this ship around, let alone reinvent it.

There’s also been discussion that student news will be largely unaffected by the tornado ripping through regional newspapers right now. Even if that is the case, I would like to propose an analogy: if you’re driving towards the cliff of irrelevance, your direction is what is most important. It doesn’t matter that your car’s engine hasn’t seized up yet.

Provide links to context, please

In an article published today, the Daily Emerald reveals that Sam Dotters-Katz has proposed changes to the ASUO Constitution (which I would link to but it’s apparently not available online):

Dotters-Katz and Papailiou’s proposal would add two justices to the existing five-member court and require it to submit rules changes to the Senate for approval. It would also re-establish the Elections Board as an independent entity to avoid conflicts of interest that Papailiou said were endemic under previous administrations.

There’s something wrong with this picture. I’m not talking about the proposed changes, rather it’s in the way that the information is presented. Reporting in the “traditional” news brief format, the reader (me) is left more confused than informed. There is almost zero context associated with the article, and I haven’t the faintest idea what the information presented actually means.

Such is the old paradigm. Newspapers are dead; long live newspapers. I’m of the opinion however that the new paradigm, the one that everyone’s afraid of, is actually improving journalism. Go figure.

For instance, if the Daily Emerald had the innovation, talent, and tools, I would have been presented an array of options to expand my knowledge about Dotters-Katz, how the ASUO runs, and why he proposes a change to the Constitution. There would likely be a list of previous posts on this issue, a small topical wiki in the sidebar synthesizing the pulse of ASUO, and curation of student blogosphere reactions to the announcement (like this one and one from the Oregon Commentator), among other forms of information.

Instead, the readers get nothing better than a press release and I have to use Google, coincidentally, to educate myself further. Google is taking the place of the news organization largely because the newspapers are flailing. Get with the times, please, and use the infinitely useful and flexible platform the internet gives you to empower your community with information.

Oh wait, the Daily Emerald runs College Publisher. Make sure your CMS is open source, and then innovate.