Wikipedia gears up for explosion in digital media

Bigger upload limits made possible by new servers, storage.

Wikipedia is gearing up for an explosion in digital content with new servers and storage designed to handle larger photo and video uploads.

Video file sizes are quickly reaching the dozens and hundreds of megabytes, and the proliferation of high-megapixel cameras means even small photos can take up a few megabytes, says Brion Vibber, CTO at the Wikimedia Foundation, which operates Wikipedia. Until early 2008, the user-generated encyclopedia's primary media file server had just 2TB of total space, Vibber says.

"For a long time, we just did not have the capacity [to handle very large media files]," he says.

Wikipedia has since scaled up from 2TB to 24TB and now 48TB of storage for its primary medial file server, and recently raised file upload limits from 20MB to 100MB. The amount of storage actually being used is about 5TB but that will grow quickly, Vibber says.

For example, users are uploading public domain classical music, and some recordings can last a half-hour. Documentary films that are out of copyright are also being uploaded, and some users are struggling to keep files under the 100MB limit, according to Vibber.

Vibber's long-term goal is to let users upload feature-length, high-quality videos, but in addition to capacity limits he says there are challenges related to getting files in the appropriate format and the physical movement of large files.

"There's no limit, and there's no practical limit," Vibber says. "The limits will get bigger and bigger to where it will be relatively easy for someone who has a legitimate need to upload a two-hour video of good quality."

Wikipedia's new servers and storage were supplied by Sun which provided donations and special pricing on Sun Fire x4500 and x4150 servers and StorageTek arrays. Wikipedia, which handles more than 10 billion page views a month, also uses Sun's open source MySQL as its primary database software.

Wikimedia operates a primary data center in Tampa, Fla., and caching centers in Amsterdam and South Korea, and has a total of 450 or so commodity servers from a mix of vendors, Vibber says.

The site's technology goals this year and next include improving usability, making it easier to edit articles and upload videos and photos, and integration with media file sites like Flickr, he says.

The donations from Sun came in the midst of a big fundraising push by Wikimedia, which raised more than US$6 million from 125,000 donors, allowing it to fund operations through the end of its fiscal year on June 30. Wikimedia is a nonprofit and the encyclopedia Web site is devoid of advertising, so the site relies on donations, grants and gifts. Wikipedia has a paid staff of 23 employees, about eight of which are in IT.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags wikipedia

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Jon Brodkin

Network World
Show Comments

Brand Post

Most Popular Reviews

Latest Articles

Resources

PCW Evaluation Team

Jack Jeffries

MSI GS75

As the Maserati or BMW of laptops, it would fit perfectly in the hands of a professional needing firepower under the hood, sophistication and class on the surface, and gaming prowess (sports mode if you will) in between.

Taylor Carr

MSI PS63

The MSI PS63 is an amazing laptop and I would definitely consider buying one in the future.

Christopher Low

Brother RJ-4230B

This small mobile printer is exactly what I need for invoicing and other jobs such as sending fellow tradesman details or step-by-step instructions that I can easily print off from my phone or the Web.

Aysha Strobbe

Microsoft Office 365/HP Spectre x360

Microsoft Office continues to make a student’s life that little bit easier by offering reliable, easy to use, time-saving functionality, while continuing to develop new features that further enhance what is already a formidable collection of applications

Michael Hargreaves

Microsoft Office 365/Dell XPS 15 2-in-1

I’d recommend a Dell XPS 15 2-in-1 and the new Windows 10 to anyone who needs to get serious work done (before you kick back on your couch with your favourite Netflix show.)

Maryellen Rose George

Brother PT-P750W

It’s useful for office tasks as well as pragmatic labelling of equipment and storage – just don’t get too excited and label everything in sight!

Featured Content

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?