Todays guest post is from Charlie Ward, VP of Platform Development at Kaplan Inc., a global educational services company that operates in 170 countries and serves one million students annually. Charlie describes why Kaplan chose Google App Engine as the foundation for a scalable, flexible, and robust education platform that allows anyone to easily deliver learning content in both live and recorded formats.The Drivers for ChangeOngoing disruptions in the Higher Education space -- including open courses, free content, regulatory pressures and protests over the increasing cost of education -- have spurred students to look for alternative paths to achieving their education and career goals and have driven institutions to look for innovative ways to lower the cost of education.
In response, Kaplan launched the KAPx initiative, whose original objective was to develop a scalable and robust platform for delivering content effectively to large audiences, using only low-cost and open components. Initial prototypes relied on traditional development technologies or required significant investments in infrastructure. When developers began to load-test these prototypes, the team recognized that Google App Engine could scale faster and further than the other products being used. So the decision was made to port the whole thing to Google App Engine.
First impressions with Google App EngineOur initial introduction to Google App Engine was a real-time commenting service that we designed around the Channel API. While load testing our prototypes, we immediately saw that Google App Engine could scale faster and further than the rest of our solution. At that point we decided to port the entire application to the Google App Engine platform.
Using HTML templates, the NoSQL datastore, and memcache for dynamic web content, three engineers with zero python experience were able to port the application and go from prototype to production in just three weeks. The team also leveraged task queues to manage message distribution and the channel API for real-time message delivery. It was App Engine’s tight integration of these components and associated Python libraries that simplified typically tedious tasks and allowed us to flatten out the learning curve.
Adding on Google Hangouts & Google AppsWith Google App Engine powering the scalable foundation for KAPx, we leveraged Google Hangouts on Air for live streaming video and built a Hangouts App for the presenter to interact with participants. We also turned to Google Apps -- Docs, Presentations, and Drawings -- and the Google Drive Picker API to allow the presenter to select, share, and incorporate lecture materials into the live experience
App Engine provided us with the tools to rapidly go from concept to production at lightning speed, without the overhead of managing a traditional infrastructure or the need to wire up disparate technologies. After starting out with live lecture delivery on
KAPx, the flexibility and ease of development on Google App Engine has allowed us to expand quickly, tackling structured learning with our first set of open courses for
Kaplan University and a completely new online institution,
Mount Washington College.
Tips for getting started with Google App Engine
- There are many excellent examples of App Engine solutions in the GoogleCloudPlatform repository on GitHub to get you started.
- Use webapp2 and the jinja2 template engine for serving up content unless you need something heavier. This lightweight but flexible combination is simple to use and provides an easy way to deliver MVC style HTML templates as well as raw data for RESTful APIs.
- Use the NDB datastore libraries to get memcache for free. NDB was experimental when we first launched our platform, so we went with GAE’s DB datastore having to write our own handlers for front loading DB queries with memcache. With NDB, this is provided for you, dramatically improving performance and shortening development time.
- Plan ahead if you are going to need to aggregate data (join, counts, etc). Operations that you would typically rely on in a relational database (joins, groupings and aggregate functions) are not as straightforward when working with the NoSQL DB or NDB datastore. You’ll want to plan out your models with the appropriate keys / properties to support aggregation in code foregoing joins and grouping when possible.
- When relying heavily on the built in datastore and memcache implementations, you’ll want to understand their upper limits and design around them. There are excellent articles available on eventual consistency, sharding data, reducing datastore contention, async task queues, and MapReduce in the developers.google.com/appengine site that can provide you with techniques to work within the boundries of service quotas and the constraints of a highly scalable environment.
- Invest time upfront to set up automated extracts of your log data to Google BigQuery on a regular basis. This will pay huge dividends the first time you need to track down a bug in production, as the dashboard tools out of the box are cumbersome to work with at scale.
- Use the --datastore_consistency_policy=random option on dev_appserver.py when developing locally to identify potential problems with delayed datastore writes before deploying.
-Contributed by Charlie Ward, VP of Platform Development, Kaplan
