Git Cherry Picking Across Forked Repos and Empty Commits

Recently I found myself in a situation where I wanted to bring in a specific upstream commit into a forked repository. Although these repos share a common history, the two repos had diverged enough that it wasn’t a straight-forward cherry-pick between branches. Instead, with clones of the two repositories I managed to cherry-pick as follows:

git --git-dir=..//.git format-patch -k -1 --stdout  | git am -3 -k

To complicate things further, a few days later, I found myself wanting to do the same thing, however, this time a submodule and another file had diverged enough that the patch no longer applied correctly. To get around this I had to:

git --git-dir=..//.git format-patch -k -1 --stdout  | patch -p1 --merge

Manually fix any of the still broken changes, then create a new commit with the changes.

These two stack overflow questions helped to work both of these issues out: https://stackoverflow.com/a/9507417 and https://stackoverflow.com/a/49537226

Finally, I’ve also in recent months found myself wanting to create a completely empty commit to kick off a downstream build process… much like you may touch a file to change its timestamp. To do this you can simply run:

git commit --allow-empty -m "Redeploy"

Merging a git repository from upstream when rebase won’t work

I use a lot of open source software in my research and work.

In recent months I’ve been modifying the source code of some of open source repositories to better suit my needs and I’ve contributed a few small changes back to the DeepLearning4J and the Snacktory projects.

This morning I’m starting to work on a further patch for the DeepLearning4J repository and I needed to bring my local repository up to date before committing the change. However, at some point over the past few months the DeepLearning4J repository has been rebased and my fork of it will no longer merge.
The usual approach for fixing this is to use the command:

git rebase upstream/master

However, for me this produces an error:

git encountered an error while preparing the patches to replay
these revisions:

As a result, git cannot rebase them.

Despite trying on two different computers similar errors occurred.

As I didn’t want to delete my entire repository and create a whole new fork of the upstream master this is the approach I took to fix the problem:

Backup the current master into a new branch:

git checkout -b oldMasterBackupBranch
git push origin oldMasterBackupBranch

Switch back to the master branch and replace it with the upstream master

git checkout master
git remote add upstream url/to/upstream/repo
git fetch hard upstream master
git reset --hard upstream/master

Push the updated master my github fork

git push origin master --force

This StackOverflow question helped a lot in working out this problem: Clean up a fork and restart it from the upstream

Git logs and commits across multiple branches

Like any good computer scientist I use git for many research and personal projects. My primary use of git is for code backups rather than collaborating with others. However, in some of my recent work I’ve been sharing repositories with colleagues and students which has caused me to improve my git skills.

The following is some of the functionality I’ve only recently discovered that has been extremely helpful:

git cherry-pick commit-id-number

This command proved very useful when I recently forked a github repo and made some changes to the source code for the specific project I’m working on. I soon discovered a bug in the original repository that a number of users had reported. I was able to fix the bug in my fork, but as my fork had changes that I didn’t want to contribute back to the original repository I was able to use the cherry-pick command to bring across only the specific commit related to the bug fix.

git checkout --theirs conflicted_file.php

Merge conflicts suck. But sometimes despite trying to pull as often as possible they still occur and can full your code with ugly messes to clean up. I recently wanted to throw away my changes to a file and simply use the latest committed file. By using git checkout –theirs I was able to throw away all my changes and go for the file that had been committed and conflicted with my changes. Conversely, you can use –ours to replace the conflicted file in favour of local changes.

git shortlog

During the past few weeks the students in the course I’ve been teaching this semester have used git to collaborate on group projects. The git shortlog command produces a list of commits grouped by each author allowing you to quickly see the relative rate at which people are contributing commits to a repository.

git branch -a

When you clone a remote repository it pulls in all branches from the remote repository. However, if you just type git branch you won’t see this. The -a flag allows you to see everything.

git log --all

The same issue applies when you are trying to see the log across all commits across all branches, just using the standard git log command will only produce the log for the current branch. The -all flag allows you to see the log across all branches, combining this with the cherry-pick command is very useful when you want to bring across just one set of changes rather than merging a whole branch.

git log --all --stat --author="Tom"

Bringing this all together I’ve begun to regularly use the above command to see all commits by a single user across all branches. This has been a good way to measure students’ contributions to a group project (note: the author option is case sensitive).

Android Blog Reader Application – Honours Assignment

One of my papers this semester is focused on Google Android Mobile Operating System.

The brief for my final assignment is:

Your task is to write any application you like. The are no restrictions on what your application can do but it should show of the capabilities of the platform and be well written.

Marks will be awarded for interesting applications that make good use of the Android platform.

Make sure your application works on the emulator but I will also test it on a real device.

For this assignment I decided to create a Blog Reader that reads the RSS XML feeds off blogs to display them in a Android Application.
I have spent around three days coding this assignment. I will not release the code until after the assignment has been marked, however here are some screen shots of the work

My Application Sitting in the Android Application Menu on my phone
My Application Sitting in the Android Application Menu on my phone
The application's home screen
The applications home screen
Adding a new blog feed into the application
Adding a new blog feed into the application
Viewing a list of blogs
Viewing a list of blogs
Removing a blog feed
Removing a blog feed
Viewing a list of blog posts
Viewing a list of blog posts
Viewing list of blog posts
Viewing list of blog posts
Viewing single post
Viewing single post

Simulating / Emulating a MPI Cluster or Supercomputer under Ubuntu Linux

Okay a few posts ago I mentioned that I had been successful in setting up my laptop as a virtual supercomputer.

Here are hopefully some relatively clear instructions on how simple it was to do.

Setting up a MPI development system on Ubuntu Linux

  1. Download Ubuntu – check out the latest version at www.ubuntu.com

  2. Intall the required mpi files.

    For Ubuntu 9.04:
    In a terminal window enter:
    sudo apt-get install libopenmpi1 libopenmpi-dbg libopenmpi-dev openmpi-bin openmpi-common openmpi-dev

    For Ubuntu 10.04:
    In a terminal window enter:
    sudo apt-get install libopenmpi1.3 libopenmpi-dbg libopenmpi-dev openmpi-bin openmpi-common openmpi-dev

  3. Test the mpi install by compiling and running a simple program

    1. mpicc testfile.c -o testfile

    2. mpirun -np 2 ./testfile

      1. Where np is the number of cores * number of processes on your system.

      2. e.g. on a two core laptop np should be 2.

  4. If you get an error regarding ssh when you enter the mpirun command install ssh

    1. sudo apt-get install ssh

  5. If you get requested for your password everytime you run mpirun set up a stored RSA key control

    1. cd $HOME

    2. ssh-keygen -t dsa

    3. cp .ssh/id_dsa.pub .ssh/authorized_keys2

Further reference:

http://www.csc.cam.ac.uk/academic/practicals.html

http://www.math.umbc.edu/~gobbert/mpi.html

Wubi

Okay, now there is no excuse for any geek or anyone with a keen interest in computers to not try out linux.

For the past year and a bit I have been playing around with Virtualbox and virtualization, the major problem here however is the speed of booting, and drivers etc as it is in an emulated/simulated environment.

Enter WUBI. Wubi is an installer that allows you to install Ubuntu and Linux Mint on an NTFS partiation and use the windows boot loader so it does absolute nothing to windows at all. And to make matters better you can access your windows files straight out of the box.

I have now had a WUBI Ubunutu set up for the past week and it is brilliant.

I am now trying a new distro called Linux Mint which is aimed at end users.

If you want some help getting it installed or just want to try it out just contact me, it is brilliant just really awesome.

Traffic Simulation Update: Multiple Cars, Multiple Lanes

Okay, the traffic simulator project has been slow going over the past few weeks, just not been very focussed and playing with DLA and other assignments too much.

This morning/arvo I changed some of the code and layout of how cars were being generated. Now instead of a car know what road it is on it has been reversed, so each road has its own collection of cars to manage. With some changes to how the graphics are produced this now means that cars are now rendered at the same time as each road. A that allows perfect placement of cars onto roads rather than having to do a mathematically approximation. Clear as mud?

carsonlanes

6000 Point Balanced DLA

Yes I am still making DLAs just trying to make the maths that little bit nicer, and get that all impressive performance boost.

This is a 6,000 point balanced model, it took just under 1.5 hours to generate, compared to 4+ hours for a 5,000 point unbalanced DLA on Sunday

6000

The C# battle of the SortedList, SortedDictionary, and List

Okay as part of my internship I have to deal with a huge number of text strings.

These strings come into the program unsorted and they must be sorted and each one must be unique. (i.e. no duplicates).

Now there are a few different ways to be able to store this data.

A SortedList, SortedDictionary, or two different forms of lists, the first where before each add you check to make sure that the data doesn’t already exist, and the second where you just add then sort then remove duplicates at a later time.

Two work out what one would be best I wrote a program to determine which form of storage operated the fastest on input. The results follow and then the code for how I did it.

As you can see the SortedDictionary worked the best, however at the early stages both the SortedList and Duplicate List gave it a run for it’s money.

Results

Sorted List Test:        00:00:00.1169883 Input:    5000    4000 List Size:    2838
Sorted Dictionary Test:  00:00:00.1339866 Input:    5000    4000 List Size:    2858
Unique List Test:        00:00:00.1119888 Input:    5000    4000 List Size:    2862
Duplicate List Test:     00:00:00.0239976 Input:    5000    4000 List Size:    2832
Sorted List Test:        00:00:01.3768623 Input:   50000   40000 List Size:   28516
Sorted Dictionary Test:  00:00:01.0678932 Input:   50000   40000 List Size:   28466
Unique List Test:        00:00:12.3097689 Input:   50000   40000 List Size:   28384
Duplicate List Test:     00:00:01.2058794 Input:   50000   40000 List Size:   28549
Sorted List Test:        00:01:36.9733017 Input:  500000  400000 List Size:  285367
Sorted Dictionary Test:  00:00:12.3307668 Input:  500000  400000 List Size:  285422
Duplicate List Test:     00:02:32.7467238 Input:  500000  400000 List Size:  285506
Sorted Dictionary Test:  00:02:37.8040000 Input: 5000000 4000000 List Size: 2854095

Continue reading “The C# battle of the SortedList, SortedDictionary, and List”

Word of the day: Reabsorbsinged

Okay I got a lession in why you shouldn’t rush code on a Friday night.

This morning on my internship I was cleaning up some code I had written last friday and making sure everything worked as planned.

While checking some of the output of the program I came accross this word: Reabsorbsinged. I thought it looked a little odd and upon closer inspection of the code I found out why.

The word Reabsorbsinged is made up of one base word: absorb. From this you can combine prefix and suffix to build more words e.g. Reabsorb, Absorbs, Reabsorbs etc.

However in my blind coding last week I had failed to realise a cruicial mistake I had made when trying to take a shortcut. I had fed my input variable into a function and overwritten it at the sametime. This is a good trick if you want to minimise memory and you don’t need to worry if you input variable is overwritten.

Unfortunately I needed my input variable to stay intact to be able to generate the other words (like above). Instead I ended up with just one huge word: re+absorb+s+ing+ed So like you end up with dick of the day in some jobs I now have word of the day.

And on an entirely different note:

I am currently converting the code that generated this mistake from c++ to c#. Easy enough C# is pretty similar to java and doesn’t have pointers, yes! However as I discovered it doesn’t have a string reverse function either.

Glancing on the internet there are a few around pretty much going from extremely long and memory expensive, i.e. copying each character onto a new string at each step, or extremely quick but near impossible to read, understand or debug. So I got smart and wrote my on.

The code was along the lines of this:


string normalString = "abcdef";
char[] tempString = normalString.toCharArray();
for(int i=0; i < tempString.length / 2; i++) {
char tempc = tempString[i];
tempString[i] = tempString[tempString.length - i -1];
tempString[tempString.length - i - 1] = tempc;
}
string reversedString = tempString.ToString();