Managing multiple versions of Java locally: SDKMAN to the rescue!

I wanted to try the newly added Records to Java 16, mostly because I immediately had the feeling that they would be pretty much the same as Kotlin data classes. As it turns out, they are pretty similar but have certain differences, but that is not the point of this entry.

As I work on projects that still depend on Java 8, I couldn't just simply upgrade my version of Java. Luckily I have been using SDKMAN on my machine, even though I initially did not have a reason to do so, but as it happens, the new setup for my tests was rather straight forward.

Adding a new version of Java is just a matter of running the following command (in my case to add Java 16):

sdk install java 16.0.1.hs-adpt

And to use it you can:

sdk use java 16.0.1.hs-adpt

But it is a bit of a pain to have to run the 'use' command every time you want to work on that project. The solution is to add a config file in the root folder of your project called `.sdkmanrc`. This is the content of my config file:

>cat .sdkmanrc
# Enable auto-env through the sdkman_auto_env config
# Add key=value pairs of SDKs to use below
java=16.0.1.hs-adpt

Finally, if you want the version in the rc file to run automatically when you cd into a particular project you can set this up in the overal sdkman config file, which on mac is at  `~/.sdkman/etc/config`. There you can change the default `false` value of the following property to true:

sdkman_auto_env=true

You can do a lot more with SDKMAN, but this was exactly what I needed for the particular tests I had in mind. Enjoy!

A very brief intro to Machine Learning for Developers

Last Saturday we ran a TensorFlow codelab at the GDG Dublin devfest17. We worked through the tensorflow for poets codelabs after a short intro to machine learning concepts, targeted towards developers, and with the only goal of making sure that the concepts in the codelabs could be correctly understood. Here are the slides from the event:



The devfest was a blast, as usual; looking forward to next year already!

Some talks and papers for 2016

It's been a really busy end quarter of the year, and in the last few months I got around to publishing one paper, and doing a number of presentations.

Back in September I was lucky enough to travel to beautiful Kraków in Poland to talk about data science and IoT-ish stuff at DevDay16.
My talk was titled 'A Homespun Decentralised DIY Data Science Research Pipeline for the Internet of *Your* Things' (quite a mouthful!), and it basically relates my adventures in the last year, trying to move from a software development background into a more data science oriented view of the world. I also rant about privacy for quite a while.

These are the slides and recording of the talk:





The conference was fantastic, flawlessly organised, and they treat speakers incredible well. You should definitely consider their CFP next year!

In October I got the chance to present at the Mobile!16 workshop at SPLASH in Amsterdam. My paper is about requirements for mobile health applications, with an End User Development flavour to it, provided by App Inventor. The paper can be found here, and the slides are:




Finally in November I gave another talk at the PyCon Ireland conference, this time about image classification in Python. During the summer a group of people helped me collect pictures of food, with a view to figure out some way of classifying the contents. There are a few companies out there such as Clarifai and Indico that do a pretty good job at classification and tagging of images (Google and Microsoft also have very good vision products, but I didn't want to go down that path). Their APIs are pretty nice and easy to use, but I wanted to figure out if I could locally reproduce at least some of the results they provide. My first attempt was to do classification with a technique called 'bag of features', which I must say, didn't work incredible well in complicated pictures such as food. After that I followed today's trend of using deep learning (everybody is at it!), more concretely using TensorFlow to retrain the Inception model that Google made available at some point last year. Results, as you can imagine, are much better, even if I can't really explain any of the math behind it!

The title of the talk was 'Introduction to Image Classification in Python: from APIs to Neural Networks', and here you can find my slides (recordings are not available yet):



Oh, and I also got a chance to finally go to the Mozilla Festival in October. We run a session about Paper Badger, but I'll tell you all about it in a different post!

App Inventor dev tricks: reducing compilation time while doing development

I want to share a few tricks that some of us use for speeding up App Inventor when we are full on development mode. Note that most of these tricks should only be used while developing, but when you are ready to open a pull request you should skip all tricks, make a full compilation for all languages and browsers, and fully test (manually as much as you can, and always run `ant tests`).

The first thing you should check (at least from time to time) is the listing of available targets in the main ant script. Here is how:

Tip 1: check out all targets available with `ant -p` from time to time.

At this particular time, it looks like the following:

appinventor-jos$ ant -p
Buildfile: ~/appinventor/build.xml
Other targets:

AIMergerApp
Blockly
BlocklyTest
PlayApp
RunLocalBuildServer
all
clean
comps
installplay
javadoc
noplay
tests

Default target: all

As you can see, there are a number of targets here, but in this post I will only go into the ones I consider that save time during development. The following are only recommendations, you do not have to do this, of course!

Also consider that if you see a new target being added, make sure you understand how to use it because it can save you some time in the long run.

Tip 2: avoid using `ant clean` whenever you can.

Compiling App Inventor is done incrementally, so only what's needed to be compiled will be compiled. Unfortunately, the GWT part of the system takes literally ages when a full compilation needs to happen (see Tip 6 for more on this). If you are making changes in that part of the system, you are kind of stuck with a full compilation, but it does not mean you need to clean all the time.

The clean target does not only delete all the compiled files, but also deletes all your projects, so use with caution. When would you use clean, then? Well, if things start to get weird, like rare errors you haven't seen before, or compilation errors that do not seem to appear in your IDE, then it's time to clean. Another good indication of cleaning time is if you change branches that contain incompatible code. But if you can avoid this, there are a number of other targets that can help you out. Let's see some of them.

Tip 3: When in Blocky Land, do as the Blocklians do.
All the App Inventor Blockly related code gets compiled into a single file called blockly-all.js that you will find in your build folder. If you are making JavaScript changes (and only in the /blocklyeditor or lib/blockly files) you can rebuild this file by calling `ant Blockly` (note the Capital B). No other files need rebuilding (generally), and this target takes time in the order of seconds and not minutes! Make sure you check out tip 4 to see how to reload this file without restarting the main server.

Tip 4: There's no need to restart the main server --- most of the time!
Most times there is no need to restart the app engine server. By disabling caching in your browser(s) and reloading the project, all should be good to go.
You can disable caching in Chrome Dev tools or in Firefox (probably in Safari too!).

In Firefox, go to 'Tools --> Web Developer --> Developer Toolbar' and click on the wrench icon at the end of the toolbar, at the bottom of the page. The toolbar will enlarge and then you can click on a gear icon for settings. Scroll down to 'Advanced Settings' and you will see an option like the one in the following figure:

Disabling cache - Firefox

In Chrome Dev Tools you simply need to click on the gear icon for settings and you will see something similar to the next figure:

Disabling caching - Google Chrome

Note that disabling cache only applies when you have the dev tools open, so you don't have to worry about your normal web surfing with this setting.

Tip 5: Component changes might not need a full build
UNLESS you change the signature of a method (in blocks this can be a method or event signature, or a property), you will not need to reflect those changes in the designer, so there's no need to build the GWT part of the system.

When no changes to the designer are needed, you can use `ant comps`, which will not only compile exclusively changes to your component, but will also push a freshly baked App Inventor Companion app to your phone if you have it connected through USB.

Tip 6: Minimize permutations in GWT compilation
There are two ways of minimizing the number of compilations that GWT will run during a full build, one being restricting the number of browsers you build for, and the second being the number of languages you fully build. These changes can be made in the YaClient.gwt.xml file.

If you generally develop only using Chrome, or Safari, or Firefox, you can modify the GWT config file to avoid building for other browsers. In line 10 in the following gist, in the value for user.agent you can choose between 'safari' and 'gecko1_8'. If you generally only use firefox, you can delete everything before the comma (including the comma). If you generally use chrome or safari, keep the safari value and delete the comma and the gecko part. This will compile GWT only for the browser you have chosen.

Since we have introduced the internationalization code, the more languages we add, the longer it takes to build (more permutations of the code). In the gist again, if you comment out lines 104 to 109, both included, you will only build the interface in English.

You can combine languages and browser and go down to 2 permutations instead of 9, which will save several minutes from your build.

Tip 7: Do you really need that build server running all of the time?
This isn't really a compilation tip, but it might also save you time while developing.
Some people use a starting script that runs both the app engine server and the build server. Most of the time (note, not all of the time!) you will be making a bunch of changes and testing them on the companion if they are component changes, or in the browser if they are interface changes. You will only build an apk towards the end of the process (in component changes) before submitting a pull request. So you won't need to spin up the build server most of the time.

NOTE
As mentioned at the top of this post, please note that most of these tips are to be used only during development, BUT before you open a pull request, you should test fully, and with all languages and all browsers.

Happy (and hopefully shorter) building!

Support for building apps on API Level 4 in App Inventor - Better Tablet Support

We've been working to take App Inventor out of screen compatibility mode, and that work has now been merged. This hasn't made it to a release yet, but it will soon. We will announce the changes in the forums when this happens.

What does it all mean though?
For starters, App Inventor will not support Android 1.5 (API level 3 - Cupcake) anymore, but we are compiling against Android 1.6 (API level 4 - Donut). We know there aren't a lot of devices out there still working at this level, but the change from level 3 to 4 was a rather large step to take, mainly because of the already mentioned screen compatibility mode.

Up to now, all App Inventor apps would be automatically scaled by the system to make them look the same in all devices, no matter their resolution or screen size. This sounds good, but the effect of this was that most elements would look rather awfully oversized on bigger screens such as tablets.  From now on, the resolution of the device will be used, and apps will look much better in larger screens with higher resolutions. As a drawback, the App Inventor designer will be a bit less in sync with what's actually being rendered on the device, but we've done our best to keep things as smooth as possible. If you find any issues with rendering in the designer please do get in touch with us through the forum.

What other changes have been made?
We have added a check to change from Tablet to Phone preview mode. It basically resizes the viewer so that you can see how the app looks at two different sizes. We are using Nexus sizes as generic here, Nexus4 for phone, and Nexus7 for tablet. As you can see in the screenshot below, we have also added the buttons bar found in most newer devices. The two changes are highlighted in red.

Another big change is that now all sizes for elements are density-independent pixels (DP) instead of hardcoded pixels. You will see a change in the Width and Height boxes in the designer.

The concept of DP is a little hard to grasp. Some people compare it to using percentages for sizes in HTML, but that's not quite correct. This is the definition used in the main Android docs:

Density-independent pixel (dp)

A virtual pixel unit that you should use when defining UI layout, to express layout dimensions or position in a density-independent way.
The density-independent pixel is equivalent to one physical pixel on a 160 dpi screen, which is the baseline density assumed by the system for a "medium" density screen. At runtime, the system transparently handles any scaling of the dp units, as necessary, based on the actual density of the screen in use. The conversion of dp units to screen pixels is simple: px = dp * (dpi / 160). For example, on a 240 dpi screen, 1 dp equals 1.5 physical pixels. You should always use dp units when defining your application's UI, to ensure proper display of your UI on screens with different densities.

This DesignBytes video by Roman Nurik is a good source of information about the topic:




There is an additional property that has also been added to Images: scaleToFit; checking this box you can make an image fit completely the size of its enclosing area, for instance if you fill the parent in width or height, the image will fit the area (the image will not keep its aspect ratio, so be mindful when using this property).


So, with all these changes, what are the things to watch out for?
Some apps might be affected, especially those using hardcoded sizes in pixels. We have made many tests and the majority show very little differences, but there might still be some differences.
The other thing to watch out for is if you were using any tricks to grab the height and width of the screen to size your elements. Watch out for problems in there and please report whatever you find.

And what's next then? Well, this change opens a number of possibilities for App Inventor apps. We can start thinking about using a targetSDK level now, even providing a property in Screen1 to allow the user to choose which level that is. We could also start thinking about moving to API Level 8 (Froyo) as there are not many devices using a level lower than that. But probably the most important change here is that we can now start using support libraries and think about adding some of the most used design patterns in Android, such as Action Bar, navigation patterns, or even themes.

This is all a bit far ahead, but there's no technical reason stopping us now from working on changes like those. If you want to help with any of these projects, do get in touch through the open source forum. Great times ahead!

Software Engineer / Research position - MIT App Inventor

The MIT App Inventor team is looking for a software engineer with interest in research in education, and, of course, open source.
You can find more information in this thread. If you want to know a little more about what we do, you can keep on reading this blog, or ping me on twitter and I'll be happy to talk to you.

What are you waiting for to apply? do it... now!

Random Hacks of Kindness Boston 2014

The fine folks of the Boston brigade of Code for America have done it again... a fantastic weekend full of hacking, delicious food, and awesome projects!

This was also part of the kickoff of the National Day of Civic Hacking competition, but back to Boston, you can find tons of information in the event's hackpad.

Here though, I'm going to write a little about the project I was involved in, Union Capital Boston. The main idea is to create a system of rewards, similar to the typical restaurant or coffee shop loyalty cards, but in which people accumulate points through good deeds in their communities. This is targeted to individuals and families within lower income brackets, so that they can exchange points for things that can help them make ends meet, such as groceries or travel passes.


On the tech side, the main app is being built on meteor. To be honest, I didn't have a lot of time to form an opinion about how meteor works, but the one thing that took me by surprise is that being a node framework, it has its own package manager, and things are a bit confusing (sometimes you load packages from meteor, sometimes from atmosphere, and there's some support for npm??? dunno, I'm confused!).

I spent most of the Saturday trying to push the meteor app into a Cordova app. After many hours fighting with a few different solutions I found on the net, it turned out that the wifi we were using wouldn't allow two devices to talk to each other; so most of my time was spent trying to fix the unfixable. On Saturday night I arrived home, compiled the app without changing a thing, and it all worked. On Sunday back at GreenTown Labs, it wouldn't work again. In any case, you can check out the app in my github repo at UCapp. The readme file also documents all the research I went through on Saturday, and some failed attempts. You can find the main UC Boston app at Duncan's repo, and a lot more information in our project's hackpad.

It was great to meet new people and see some old friends that I hadn't seen in a while. Looking forward to the next meetup (and hackathon) already!

Closing #RHoKboston @carpeliam and @_beechnut pic.twitter.com/Ta11CcAIqO
— Jos (@josmasflores) June 1, 2014