Test Drive Evaluation

Test Driven Evaluation

To better explain what Test Drive Evaluation (TDE) is let’s first look at what the principle is based upon…Test Drive Development (TDD)

Source:

Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or 'rediscovered' the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.[1]

Test-driven development is related to the test-first programming concepts of extreme programming, begun in 1999,[2] but more recently has created more general interest in its own right.[3]

Programmers also apply the concept to improving and debugging legacy code developed with older techniques.[4]

Emphasis mine

Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or ‘rediscovered’ the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.

With the growth, popularity and increased acquisitions of BaaS providers it becomes harder and harder for newcomers to properly evaluate which one will provide the best value added to their application. Please note that I didn’t specify mobile application. While I personally subscribe to the “mobile first” approach…native desktop applications and traditional web applications shouldn’t/can’t be seen as second class citizens, thus BaaS should provide support for those as well.

Myself and many others have written on the pro’s / con’s of the top BaaS players. Unforunately, there hasn’t been a set of standard tests that could be run and give a definitive “pass or fail” result for the features that the providers offered. The KitchenSync project aims to provide the tests for these features.

Everything and the KitchenSync

It doesn’t matter if you’re a developer, project manager or analyst there is never enough time to implement every feature, RTFMs or write POC’s (when evaluating different vendors, technology, etc) for an application. We tend to cut corners, take for face value what is on the brochure or go with a solution just because it was featured as the latest/greatest startup on Techcruch.

For example:

  • What does it take implement object level security?
  • Does the BaaS provide static file / webhosting?
  • Which providers will allow full text searching?
  • How many API calls to it take perform request abc

What is the Best Option?

Antonio Martinez does a great job of summarizing:

Source: http://www.raywenderlich.com/20482/how-to-choose-the-best-backend-provider-for-your-ios-app-parse-vs-stackmob-vs-appcelerator-cloud-and-more

The most important factor here is to evaluate each service’s features and their usefulness to you — and your app! Preconfigured objects might seem attractive, unless your idea of what those objects should be differs from those of the service provider.

Services that offer just “users” along with general purpose objects might initially seem more difficult, until you see that substantial flexibility comes with the user-based approach.

Will you need a support agreement; and should you be paying for phone support? How accessible are the engineers if you need them, and what kind of user community support can you expect?

Take a close look at the documentation. This is one time you should read it thoroughly, even though the sample projects have given you enough to get started! :] The quality, coverage and currency of the documentation will give you a good idea of how committed each company is to your success.

Make yourself aware of the costs you will incur in using the service. This will require you to project your future usage for best case and worst case scenarios. Should you find yourself with 100 million users, will your app generate enough income to pay for the level of service you will require?

If your app is going to require a paid plan from the start but it flops, can you afford that? Some services require you to contact sales before you know what these costs will be; don’t skip this step!

Doing a search for the name of your service on discussion boards might expose interesting bits of information you might not have found otherwise. If you get no hits at all, how popular is the service — and what are the chances it will be around in six months? If all the hits say “it’s awesome” that might be a good indication – but pay attention to the negative comments as well!

You can arrive to most of the answers just by reading the documenation, but where the 1’s and 0’s meet the compiler is at implementation. Enter the KitchenSync project.

Even with my own evaluations and recommendations I wanted to give developers, analysts and just the curious, a set of common unit tests to run against each of the providers so that informed decisions could be made with the utmost transparency, while providiing a forum for feedback, additions and bug fixes.

All the tests are open for anyone to use for your own evaluations - just download and run them. In addition, as mentioned above, anyone is welcome to join the party - all enhancements, corrections and suggestions are welcome. There are a number of additional vendors as well - and I invite them, or any developers familiar with them to contribute implementations for them as well and I will be happy to add them to the report as time permits or I can get to them.

You Forgot About Feature ‘X’ and Provider ‘CDW’

No, I didn’t. There had to be a version 1.0 so I chose the providers that I’ve used (Parse, Stackmob, Kinvey and FatFractal) and a basic set of tests. I will be adding more tests just as quickly as I can write them, but I’m hoping for as many pull-requests as I can get. Specifically for new providers.

Disclaimer

I am not employed by any of the providers nor paid, bribed or coerced into writing this article or performing the tests nor is this an endorsement of my employer. I wrote it because…well…no one else had.

Applicasa As Your BaaS

The New Comer – Applicasa

Last month I wrote a quick review of the two BaaS providers that I had used, StackMob and Parse. A few days later I received an email from Tzvi Kopetz, COO and co-founder of Applicasa, a backend solution provider who, even in it’s infancy, is filling in some of the feature gaps that Parse and StackMob have left open. Three of the biggest ones are managing AdHoc builds, a dedicated CMS and custom app framework.

Adhoc Builds

Seeing Applicasa getting closer and closer to provide the Adhoc capabilities of HockeyApp or TestFlight is an evolution that all mobile developers should pay attention to. I currently, have to manage my data/backend api with another provider while managing adhoc builds with TestFlight. Having everything centrally located would be a huge win.

CMS

I am a developer and really don’t want to worry about creating content. At a minimum someone could talk me into writing a script that populates schemas via script or upload. However, most other platforms don’t provide the necessary environment for a content editor role and a CMS that doesn’t have the unnecessary bloat of a traditional CMS.

With Applicasa you are able to create content editors that have a completely separate role from developers so they have an easy and intuitive console for managing the content. Both Parse and StackMob have content consoles with some management capabilities, but they aren’t as featured and you can tell they want you to populate your data via the API.

You can use backend service to send versions (updates) to different clients, use our CMS. I think our CMS is quite a > different approach from the other providers. We actually give you a complete solution so you don’t have to build a CMS, you don’t have to build a complete system for your clients (users). You can actually invite clients to use the system with the developers. – Lior Malenboim, CEO

Source

Custom Framework

When it came time to actually integrate their framework into my test app I kept looking for a github repo or direct framework download from their site, but couldn’t find it. The reason being is that the framework is generated, by the developer on the fly based upon your database schemas and relationships. This is a feature that I never really thought about. Probably because I haven’t seen anyone do this. Parse and StackMob do a great job with their frameworks in regards to abstracting models on the client, however, they don’t give this fine grained control. The ability to have this type of control over the framework, particularly the models, is because the Applicasa framework is generated by the developer and stubbed out based upon your exact data structure.

Summary

Applicasa is a great backend provider with amazing features and support who is truly shaping up to be a “one stop shop”. I have talked with their founders and developers a few times over the past month and they have answered all my support questions promptly and notified me of fixes to any issues I discovered. Over the past month they have rolled out and/or updated their customized objects and iOS SDK, CMS, Webservice with code snippet, Push Notifications, Ad Hoc beta versions, Users management and tickets management system. I am looking forward to seeing what they come out with next.