Git Cherry Picking Across Forked Repos and Empty Commits

Recently I found myself in a situation where I wanted to bring in a specific upstream commit into a forked repository. Although these repos share a common history, the two repos had diverged enough that it wasn’t a straight-forward cherry-pick between branches. Instead, with clones of the two repositories I managed to cherry-pick as follows:

git --git-dir=..//.git format-patch -k -1 --stdout  | git am -3 -k

To complicate things further, a few days later, I found myself wanting to do the same thing, however, this time a submodule and another file had diverged enough that the patch no longer applied correctly. To get around this I had to:

git --git-dir=..//.git format-patch -k -1 --stdout  | patch -p1 --merge

Manually fix any of the still broken changes, then create a new commit with the changes.

These two stack overflow questions helped to work both of these issues out: https://stackoverflow.com/a/9507417 and https://stackoverflow.com/a/49537226

Finally, I’ve also in recent months found myself wanting to create a completely empty commit to kick off a downstream build process… much like you may touch a file to change its timestamp. To do this you can simply run:

git commit --allow-empty -m "Redeploy"

Docker CFLAGS march=native and illegal hardware instruction

I’ve recently hit an interesting issue involving Docker containers and compiled C code crashing code with illegal hardware instruction errors.

In a nutshell the Docker image was built on a server using Jenkins. It has been running fine until the underlying host orchestration software and physical hardware changed.

My solution has been to remove the build of the C code from the Dockerfile and instead put in inside a script which runs when the container starts.

For my use case this solution will work, however, it will require the C code to be compiled every time the Docker container runs.

Digging a little deeper, it appears that the underlying root issue may be the use of the CFLAGS march=native flag in the C code’s underlying Makefile.

A few posts on Stack Overflow (in particular, https://stackoverflow.com/questions/54039176/mtune-and-march-when-compiling-in-a-docker-image), suggest that removing this flag may make little difference to the run time of the compiled code. Which, again depending on the use case, may be a good option to try as well to resolve these problems.

Merging a git repository from upstream when rebase won’t work

I use a lot of open source software in my research and work.

In recent months I’ve been modifying the source code of some of open source repositories to better suit my needs and I’ve contributed a few small changes back to the DeepLearning4J and the Snacktory projects.

This morning I’m starting to work on a further patch for the DeepLearning4J repository and I needed to bring my local repository up to date before committing the change. However, at some point over the past few months the DeepLearning4J repository has been rebased and my fork of it will no longer merge.
The usual approach for fixing this is to use the command:

git rebase upstream/master

However, for me this produces an error:

git encountered an error while preparing the patches to replay
these revisions:

As a result, git cannot rebase them.

Despite trying on two different computers similar errors occurred.

As I didn’t want to delete my entire repository and create a whole new fork of the upstream master this is the approach I took to fix the problem:

Backup the current master into a new branch:

git checkout -b oldMasterBackupBranch
git push origin oldMasterBackupBranch

Switch back to the master branch and replace it with the upstream master

git checkout master
git remote add upstream url/to/upstream/repo
git fetch hard upstream master
git reset --hard upstream/master

Push the updated master my github fork

git push origin master --force

This StackOverflow question helped a lot in working out this problem: Clean up a fork and restart it from the upstream

Git logs and commits across multiple branches

Like any good computer scientist I use git for many research and personal projects. My primary use of git is for code backups rather than collaborating with others. However, in some of my recent work I’ve been sharing repositories with colleagues and students which has caused me to improve my git skills.

The following is some of the functionality I’ve only recently discovered that has been extremely helpful:

git cherry-pick commit-id-number

This command proved very useful when I recently forked a github repo and made some changes to the source code for the specific project I’m working on. I soon discovered a bug in the original repository that a number of users had reported. I was able to fix the bug in my fork, but as my fork had changes that I didn’t want to contribute back to the original repository I was able to use the cherry-pick command to bring across only the specific commit related to the bug fix.

git checkout --theirs conflicted_file.php

Merge conflicts suck. But sometimes despite trying to pull as often as possible they still occur and can full your code with ugly messes to clean up. I recently wanted to throw away my changes to a file and simply use the latest committed file. By using git checkout –theirs I was able to throw away all my changes and go for the file that had been committed and conflicted with my changes. Conversely, you can use –ours to replace the conflicted file in favour of local changes.

git shortlog

During the past few weeks the students in the course I’ve been teaching this semester have used git to collaborate on group projects. The git shortlog command produces a list of commits grouped by each author allowing you to quickly see the relative rate at which people are contributing commits to a repository.

git branch -a

When you clone a remote repository it pulls in all branches from the remote repository. However, if you just type git branch you won’t see this. The -a flag allows you to see everything.

git log --all

The same issue applies when you are trying to see the log across all commits across all branches, just using the standard git log command will only produce the log for the current branch. The -all flag allows you to see the log across all branches, combining this with the cherry-pick command is very useful when you want to bring across just one set of changes rather than merging a whole branch.

git log --all --stat --author="Tom"

Bringing this all together I’ve begun to regularly use the above command to see all commits by a single user across all branches. This has been a good way to measure students’ contributions to a group project (note: the author option is case sensitive).

Android Blog Reader Application – Honours Assignment

One of my papers this semester is focused on Google Android Mobile Operating System.

The brief for my final assignment is:

Your task is to write any application you like. The are no restrictions on what your application can do but it should show of the capabilities of the platform and be well written.

Marks will be awarded for interesting applications that make good use of the Android platform.

Make sure your application works on the emulator but I will also test it on a real device.

For this assignment I decided to create a Blog Reader that reads the RSS XML feeds off blogs to display them in a Android Application.
I have spent around three days coding this assignment. I will not release the code until after the assignment has been marked, however here are some screen shots of the work

My Application Sitting in the Android Application Menu on my phone
My Application Sitting in the Android Application Menu on my phone
The application's home screen
The applications home screen
Adding a new blog feed into the application
Adding a new blog feed into the application
Viewing a list of blogs
Viewing a list of blogs
Removing a blog feed
Removing a blog feed
Viewing a list of blog posts
Viewing a list of blog posts
Viewing list of blog posts
Viewing list of blog posts
Viewing single post
Viewing single post

The C# battle of the SortedList, SortedDictionary, and List

Okay as part of my internship I have to deal with a huge number of text strings.

These strings come into the program unsorted and they must be sorted and each one must be unique. (i.e. no duplicates).

Now there are a few different ways to be able to store this data.

A SortedList, SortedDictionary, or two different forms of lists, the first where before each add you check to make sure that the data doesn’t already exist, and the second where you just add then sort then remove duplicates at a later time.

Two work out what one would be best I wrote a program to determine which form of storage operated the fastest on input. The results follow and then the code for how I did it.

As you can see the SortedDictionary worked the best, however at the early stages both the SortedList and Duplicate List gave it a run for it’s money.

Results

Sorted List Test:        00:00:00.1169883 Input:    5000    4000 List Size:    2838
Sorted Dictionary Test:  00:00:00.1339866 Input:    5000    4000 List Size:    2858
Unique List Test:        00:00:00.1119888 Input:    5000    4000 List Size:    2862
Duplicate List Test:     00:00:00.0239976 Input:    5000    4000 List Size:    2832
Sorted List Test:        00:00:01.3768623 Input:   50000   40000 List Size:   28516
Sorted Dictionary Test:  00:00:01.0678932 Input:   50000   40000 List Size:   28466
Unique List Test:        00:00:12.3097689 Input:   50000   40000 List Size:   28384
Duplicate List Test:     00:00:01.2058794 Input:   50000   40000 List Size:   28549
Sorted List Test:        00:01:36.9733017 Input:  500000  400000 List Size:  285367
Sorted Dictionary Test:  00:00:12.3307668 Input:  500000  400000 List Size:  285422
Duplicate List Test:     00:02:32.7467238 Input:  500000  400000 List Size:  285506
Sorted Dictionary Test:  00:02:37.8040000 Input: 5000000 4000000 List Size: 2854095

Continue reading “The C# battle of the SortedList, SortedDictionary, and List”

Word of the day: Reabsorbsinged

Okay I got a lession in why you shouldn’t rush code on a Friday night.

This morning on my internship I was cleaning up some code I had written last friday and making sure everything worked as planned.

While checking some of the output of the program I came accross this word: Reabsorbsinged. I thought it looked a little odd and upon closer inspection of the code I found out why.

The word Reabsorbsinged is made up of one base word: absorb. From this you can combine prefix and suffix to build more words e.g. Reabsorb, Absorbs, Reabsorbs etc.

However in my blind coding last week I had failed to realise a cruicial mistake I had made when trying to take a shortcut. I had fed my input variable into a function and overwritten it at the sametime. This is a good trick if you want to minimise memory and you don’t need to worry if you input variable is overwritten.

Unfortunately I needed my input variable to stay intact to be able to generate the other words (like above). Instead I ended up with just one huge word: re+absorb+s+ing+ed So like you end up with dick of the day in some jobs I now have word of the day.

And on an entirely different note:

I am currently converting the code that generated this mistake from c++ to c#. Easy enough C# is pretty similar to java and doesn’t have pointers, yes! However as I discovered it doesn’t have a string reverse function either.

Glancing on the internet there are a few around pretty much going from extremely long and memory expensive, i.e. copying each character onto a new string at each step, or extremely quick but near impossible to read, understand or debug. So I got smart and wrote my on.

The code was along the lines of this:


string normalString = "abcdef";
char[] tempString = normalString.toCharArray();
for(int i=0; i < tempString.length / 2; i++) {
char tempc = tempString[i];
tempString[i] = tempString[tempString.length - i -1];
tempString[tempString.length - i - 1] = tempc;
}
string reversedString = tempString.ToString();

JOGL Rocks!

For the past few weeks I have been thinking seriously about heading back to uni to complete my honours degree.

Currently some of the team in Computer Science at Massey have been playing with JOGL.

I had a look at it the other day but didn’t get very far. Well tonight I decided to really get my hands dirty.

And it rocks! Im currently seeing if I can convert my traffic simulator to JOGL.

Will keep this blog posted on updates.

Two domains, one site, different pages served

I am currently working on a complex website that requires two different companies on two different domains to share the same website. I have just mananged to create a piece of php code to analyse the domain name typed in and redirect to the correct section of the site for that company. Awesome!

$url = $_SERVER['HTTP_HOST'];
if(strpos( strtolower($url), "site2" )!=FALSE) {
header('location:site2/index.php');
} else {
header('location:site1/index.php');
}

It is not often that code makes me happy.

New Year, New Website

I have just rolled out a complete overhaul of my website.

The changes in summary:

New:

  • Blog comments are back – provided I don’t get too much spam they will stay.
  • Java Web Start for software section
  • Text versions of many articles

Updated:

  • About me page
  • Articles portfolio
  • Website portfolio
  • Software portfolio
  • Photos

Deleted:

  • Homepage – Replaced with blog
  • Outdoor Activities
  • Links – Blogroll will be its replacement
  • Other stuff I can’t remember