Arch Linux with Secure Boot

Arch Linux is a great Linux distribution for those that have dipped their toes into mainstream Linux for a few years and you want to get into the full Linux experience. I took the plunge after years of happy Pop!_OS experience. I still recommend this for almost everyone, especially for a laptop with a discrete GPU. But if you're confident in Linux, its time to try the peak open source experience, so hopefully the below guide is helpful.

Arch Linux is generally not too bad to install with the 'archinstall' program after running the live boot. But for gaming, the latest games will need secure boot and hence this guide. 

Getting secure boot to work with Linux is a bit tricky needing a few videos and some trial and error to learn and get working, especially with Arch Linux because there are post installation steps that need to be completed.

You will need to decide whether you want to use secure boot, I personally did want to use it for added security. To do this, I ensured I had secure boot enabled and put it into 'Setup Mode' ready for post installation steps after a archinstall. I had an MSI board, so the following video was helpful in working out how to put the BIOS and secure boot into Setup Mode before I booted into the live image:
Setting Up Secure Boot on Arch Linux Using sbctl

Then one can boot into a USB live image with Arch Linux built with RUFUS.

This will take you into an Arch Linux terminal as root. This video has some nice explanation around it that I used including making sure to sign the repository to be used with GPG:
Arch Install and Hyprland setup

You can also follow the steps above for setting up Arch Linux install using the archinstall script/program. I chose to partition my drives in /boot, swap, /, and /home. I chose KDE Plasma desktop as the type of install. I made sure to choose the correct graphics drivers and timezone etc. I chose systemd for the boot loader because this seemed easiest and enabled following instructions straight forward.

Upon finishing install, you will be asked to reboot, do post install terminal etc. Whether you decide to have a secure boot system means you will need to follow additional steps. Not doing so will give the error

error: prohibited by secure boot policy

Secure boot setup is important for security. At this point if you have secure boot disabled in BIOS, a reboot after installing will work fine. If secure boot is enabled, you need to drop to post install terminal instead of booting immediately.

I used the following steps in this video to setup secure boot properly after the install:
Install Secure Boot on Arch Linux (the easy way)

This page is a good reference, also used in the videos above, for the entire secure boot process and signing the boot images etc. This page was also very useful when I tried using GRUB for my loader and caused the error I mentioned above.

When I rebooted, everything works as intended.

Cheers Shakes - L3mming

Setting Up Linux for Competitive Gaming (Its Mostly Simple!)

I just wanted the fastest/easiest setup that ensures getting something like Counter-Strike Global Offensive setup with mouse acceleration off and DPI settings correct.

I went with Pop OS! as my distro because it has either AMD or NVIDIA drivers bundled with the ISO image used for installation. Then use something like Rufus to create a bootable USB drive.

Once your Linux distro is installed and assuming you're using Gnome-Shell (the default window manager for Pop OS, Ubuntu and Fedora), you want to install gnome tweak tool and under mouse, change the profile to flat to disable acceleration.

To get DPI settings to match your previous settings, I used razercfg to configure the mouse settings.

Then just install Steam and Lutris from the Pop Store and you're good to go!

EDIT: You can install Discord and other commercial applications from the SnapStore.

Cheers Shakes - L3mming

Scientific Python Environment on Windows with Tensorflow

Google has made Tensorflow quite easy to install with GPU support, but getting it working along side SciPy and Numpy in a single Windows environment can be tricky. This is usually because Scipy looks for a specific build of Numpy that uses Intel MKL. This guide will allow you to set it up easily.

Firstly, I setup WinPython on my machine. You get Scikit packages, Jupyter, Scipy, Numpy, Spyder etc. in simple easy distribution that doesn't have to be installed but kept side-by-side with multiple setups. Avoid the "Zero" builds that have minimal packages. I used Python 3.5 Qt 5 installer.

Second, go to the Tensorflow website and determine the CUDA version you need and then install CUDA of that version. If you install the incorrect version, you'll keep getting the error:

ImportError: Could not find 'cudart64_90.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 9.0 from this URL: https://developer.nvidia.com/cuda-toolkit

Here, I installed CUDA 9.1 when instead the Python Package Index version of Tensorflow (v1.5 at the time of writing) required 9.0. Afterwards, install the cuDNN that matches the CUDA you have installed. This requires a sign up before login at the time of writing. Older versions of Tensorflow did not require cuDNN installed, but this is now compulsory.

Third, install tensorflow-gpu via the WinPython Command Prompt by issuing the usual command that Google recommends:

pip install --upgrade --ignore-installed tensorflow-gpu

This will give you a working Tensorflow, Numpy environment, but Scipy will be broken giving the error:

    from scipy.linalg import _fblas

ImportError: DLL load failed: The specified module could not be found.

This is where you will need to install the Numpy MKL version, which you can find here. Then use the WinPython Control Panel to upgrade the Numpy package. Make sure to download the Numpy MKL wheel to match your Python distribution.

EDIT: If you choose the version of WinPython released just after the Tensorflow release you desire, then most of the libraries will be in-sync with the Tensorflow PIP package and therefore won't give you the issues with Scipy (and other libraries) above.

That's it!

Cheers Shakes - L3mming

Setting up a Mac OSX Development Environment for VTK, ITK, Qt 4 and SMILI

I recently got access to a Mac and decided to improve support for SMILI on Mac OSX. This required setting up the Mac OSX development environment and so I log my experiences/steps in this post.

Firstly some notes. This is to setup VTK 5 and/or 6, ITK 4.10+ and Qt 4 with the current version of Mac OSX at the time - Sierra (10.12).

The first steps involved setting up Homebrew. This allows one to install Git and CMake for the command-line.

brew install make

Next we installed Xcode from the AppStore. You should not run CMake on libraries until Xcode has been installed.

Then (or during the XCode install) we download the required libraries for your environment:

• Insight Toolkit (ITK) - I used version 4.10.1, but newer versions of ITK 4 should be OK.
• Visualisation Toolkit (VTK) - I used version 6.3.0, but also a modified version of 5.8.0 (for MSVC 2013) that I made available here. Versions newer than 6.3.0 had visualisation bugs introduced into SMILI.
• Qt 4.8.x - Just use the home-brew version described here. I tried using version 4.8.6 that you can find here. Download the .tar.gz version for Mac OSX. But it had issues, see below. So the HomeBrew version as described here.

Qt 4

I Failed: After Xcode was installed, I began compiling Qt. Turns out that using the zip version messes up the line endings and breaks the configure script. So make sure to download the TAR version. When compiling, There was a compile error for the 'qpaintengine_mac.cpp' file. The fix for Qt 4.8.6 is here. There are issues with 10.9 and libstd++ which I haven't figured out yet. I will update this side of the post once I have. I will leave the configure command I tried using here for future reference:

./configure -fast -release -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk -DMACOSX_DEPLOYMENT_TARGET=10.9 -arch x86_64

You could also try 4.8.7 from here. It required the same fix as 4.8.6 from here.

I Succeeded: by using the HomeBrew Qt4 version as described here. I just did step 2 as I had a fresh install. No compilation, just downloaded and installed. This will become outdated at some point unless maintained, so you may need to search for other recipes. This link might help in that case.

ITK

No issues. Just build fine from source. I usually enable review to get access to less known ITK classes. I used the following CMake command to generate the project for Sublime Text:

ccmake -G "Sublime Text 2 - Unix Makefiles" -DCMAKE_OSX_SYSROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/ -DCMAKE_OSX_DEPLOYMENT_TARGET=10.6 ..

VTK

I Failed: Once Qt 4 is built and installed, I enabled VTK_USE_QT to ON. I had two major issues. The first issue I had was with the duplicate symbols error with TIFF. Installing lib tiff with HomeBrew solved the problem, but could potentially caused crashes in certain versions of Mac OSX, so will post a better solution later. The second was a garbage collection compilation error that I have yet to find solution for.

I succeeded: Compiling VTK 6.3 had no issues at all. I used the same CMake command as for ITK. I used deployment target of 10.6 to ensure support for Mac OSX 10.6.

EDIT: I succeeded: Compiling VTK 5.8 by ensuring that the VTK_REQUIRED_OBJCXX_FLAGS was empty. This was because of a 'garbage collection not found' error. This is because GC memory management model has been deprecated and removed. See the COCOA VTK page for more details.

SMILI

Once the above dependencies are built, SMILI v1.01 had no issues. I reused the CMake command for ITK above. I built both 10.6 and 10.9 deployment targets for ITK, VTK and SMILI for testing.

More will be added as I encounter/fix issues.

Cheers Shakes - L3mming

Paperwhite eBook Reader Experiences for Research Papers and Maths

I have been wanting an easier way to read journal papers for a while and always thought about getting an eBook reader, since e-ink should be suitable right?

Well, reading a number of reviews and guides, it seems previous e-book readers didn't seem to handle PDFs well and most times didn't handle maths well either. That's why most guides etc. recommend tools like K2pdfopt to convert PDFs. But this year I though I'd give it a shot and test to see if the myths were true and I'm happy to report that it is not true for the new models, PDFs work great and maths are flawless.... at least for the Amazon Paperwhite 2015 model.

Here's excerpts from my thesis (available open access here) displayed on the device. ;)

Portrait Mode

You can zoom like on any other tablet, with much slower update rates, but not a major drama since once you get the view you want, you can read it like paper.

Portrait Mode Zoomed

I usually use the landscape mode, where each page is shown in three sections.

Landscape mode

Surprisingly, the default PDF behavior was better than the converted PDF files were. Battery life is about a month and the reader also has back illumination for night reading, though would preferred a night mode (white text and black background) like some apps on Android tablets, but no biggy. I did find using the free Calibre to rename and reorder PDFs a must as the reader doesn't read the PDF meta data, so the PDFs are shown with filenames of the PDFs on the home screen. Adding sub-directories would be nice too on the home screen.

All in all, I would highly recommend the reader to scientists for reading scholarly articles and properly displaying mathematics.

Cheers Shakes - L3mming

VTK 5 and FFMPEG shenanigans on Windows

Over the years, trying to get VTK 5 to work with FFMPEG has always been tricky. Thankfully, VTK 6 has no such problems, but for those that have legacy code that use VTK 5, this is still a problem.

In this post I will cover some necessary tweaks, steps and links to getting FFMPEG IO working with your project that uses VTK 5, such as the 'Animate' plugin for sMILX and SMILI.

Firstly, I have found VTK 5.8 to be far more stable and workable than 5.10.1. In my scientific visualisation software SMILI, certain features don't work at all with the same code that runs fine on VTK 6 and 5.8.

Secondly, it seems VTK 5 has issues with Visual Studio 2013 (required to support Windows 8), but a patched version of VTK 5.10.1 available on GitHub works OK, though I had to fix a few round brackets here and here.

For version 1.2 of FFMPEG, this is what I recently did to get VTK 5.8 working with FFMPEG. The best link for FFMPEG deprecation changes are found here. Summarised, I had to fix vtkFFMPEGWriter.cxx as the following:

1. av_set_parameters  - comment out
2. URL_WRONLY to AVIO_FLAG_WRITE
3. url_fopen to avio_open and url_fclose to avio_close
4. av_write_header to avformat_write_header, last arg to NULL

EDIT: You may also need AVCodecID, PKT_FLAG_KEY is replaced by AV_PKT_FLAG_KEY and replacing CODEC_TYPE_VIDEO with AVMEDIA_TYPE_VIDEO.

EDIT 2: I have created a GitHub VTK 5.8.0 fork with these changes applied.

EDIT 3: GCC 9.1 causes issues. It can be fixed by setting CMAKE_CXX_FLAGS as  -fpermissive -std=c++98 -DGLX_GLXEXT_LEGACY

No changes are required for VTK 6.1.0 at the time of writing.

Hope that helps
Cheers Shakes - L3mming

Mesh/Surface Processing with Command-line SMILI tools

I recently had to do some processing for a prostate segmentation scheme I've been working on and thought some uses of SMILI tools would be helpful to others. Majority of surface processing tasks can be achieved with the milxModelApp in SMILI, a easy to use open source medical imaging library and viewer.

My tasks involved involved:

1. copying scalars from an atlas mesh to lots of other meshes
2. using scalars to clip the meshes
3. computing the surface distance of the meshes to their corresponding labelled images that were created by clinical experts
4. computing the scalar statistics across the population of meshes (my meshes already have point to point correspondence) 

Luckily, the milxModelApp application provides most of these without needing to script for multiple surfaces. Furthermore, the Qt framework is not required for these applications as they are VTK/ITK only applications (i.e. built on milxSMILI, the GUI-independent sub-library of SMILI).The following examples are for Version 0.998 (SMILI Release Beta 1).

Each one of the above tasks can be achieved as:

1. The --scalarcopy  option in milxModelApp provides a way to do this. I did:
   

   milxModelApp --scalarcopy focus_atlases/focus_bladder_atlas.vtk results/bladder/asm_bladder_*.vtk -p results_scalars/bladder/

   Copy scalars from atlas mesh (focus_bladder_atlas.vtk) to other meshes in directory (results/bladder/asm_bladder_*.vtk) and output to another directory with same names.
2. The --clip  option in milxModelApp provides a way to do this. I did:
   

   milxModelApp --clip 1 results_scalars/bladder/asm_bladder_*.vtk -p results_clipped/bladder/

   Clip meshes in directory keeping only parts with value of 1 and output to another directory with same names.
3. We need to write a script for the milxHausdorffDistance application that provides this using distance maps in a straight forward way. This script is now in the repository as scripts/ batch_hausdorff.py. An example usage of the application:
   

   milxHausdorffDistance results_clipped/bone/asm_bone_4.vtk -p Hausdorff/bone/bone__ -o Hausdorff/bone/bone__004.vtk --label manuals_renamed/bone/manual_case_bone_004.nii.gz --labelvalue 1 -c 004

   The distance map of the manual is created and the distances stored on the mesh thus giving the surface distances to the manual.
4. The --scalarstats option in milxModelApp provides this. I did:
   

   milxModelApp --scalarstats Hausdorff/bladder/bladder__*.vtk -o bladder_stats.vtk

   Compute scalars stats (mean, variance etc. per point) of meshes in directory and output single mesh (of first mesh) with stats as arrays in mesh. You can then use sMILX viewer to view the meshes and change the loaded arrays via the Right Click->Show->Load Array menu.

Note that the milxDeformableModel app is an application derived from milxModelApp and additionally provides options for voxelising surfaces and applying image orientation to surfaces.

Hope this helps
Cheers Shakes - L3mming

Compiling VTK 5.8.0 on Windows 64 with Visual Studio 2013

This is a quick post to cover the steps necessary to compile VTK 5.8.0, the most stable VTK 5 version in my experience, on Windows x64 with VS 2013 (to support Windows 8.1). The information is on the internet but scattered about so I thought I'd collect it just in case its useful for someone.

EDIT: Note that VTK 5.8.0 builds out of the box on Ubuntu 16.04 still, if you ensure that you uncomment the line with the define for GLX_GLXEXT_LEGACY in the Rendering/vtkXOpenGLRenderWindow.cxx file.

There are a few things that are incompatible with the new compiler. Firstly, the vtkOStreamWrapper error. This is covered in great detail here. Basically replace the offending line with:

//VTKOSTREAM_OPERATOR(ostream&);
vtkOStreamWrapper& vtkOStreamWrapper::operator << (ostream& a) {
    this->ostr << (void *)&a;
    return *this;
}

Then there are the ifstream->read() errors. Replace the

if ( this->IFile->read(result, 80) == 0)

with the

if ( this->IFile->read(result, 80).fail())

Lastly, there are the make_pair errors, solved here. You need to replace the

this->Map->insert(vtkstd::make_pair< vtkVariant, vtkVariant >(from, to));

with

this->Map->insert(vtkstd::make_pair(from, to)); // Addendum

in the vtkMapArrayValues.cxx file. You might need to add the include:

#ifdef _WINDOWS
  #include  // Addendum for vs11 to find 'greater'
#endif

to vtkAdjacencyMatrixToEdgeTable.cxx. I also had to add the above include and 

#include 

to vtkAdjacencyMatrixToEdgeTable.cxx, vtkNormalizeMatrixVectors.cxx, vtkPairwiseExtractHistogram2D.cxx, vtkParallelCoordinatesRepresentation.cxx, vtkChartXY.cxx, vtkControlPointsItem.cxx, to get rid of std max and min errors, 

HTHCheers Shakes - L3mming

HP 4000b Bluetooth Mouse and Ubuntu

This quick post cover howto get the HP 4000b Bluetooth mouse working flawlessly with Ubuntu 14.10 + (and maybe even older versions). These bluetooth mice are useful for ultra-portable devices that have no or few USB ports, like my Dell XPS 13, keeping the port free for other uses.

The HP 4000b mouse is an affordable bluetooth mouse that does work out of the box with Ubuntu but drops out after use for a few minutes or so. This post covers the solution to this problem that worked for me, and this post helpful for TLP users.

In summary, add the following to your rc.local file (the one I used was /etc/rc.local):

# Prevents the Bluetooth USB card from getting reset which disconnects the mouse
BTUSB_DEV="8087:07dc"
BTUSB_BINDING="$(lsusb -d "$BTUSB_DEV" |
    cut -f 1 -d : |
    sed -e 's,Bus ,,' -e 's, Device ,/,' |
    xargs -I {} udevadm info -q path -n /dev/bus/usb/{} |
    xargs basename)"


echo "Disabling autosuspend for Bluetooth USB Soundcard: $BTUSB_BINDING..."
echo -1 > "/sys/bus/usb/devices/$BTUSB_BINDING/power/autosuspend_delay_ms"

And in the TLP config (/etc/default/tlp), add the USB_BLACKLIST="8087:07da". The ID for my device was 8087:07dc as in the rc.local file. These IDs need to match and can be found using lsusb.

HTH
Cheers Shakes - L3mming

Visualizing 32-bit Integer PGMs (eg. from FTL) or other ASCII Image Formats

In this post I will post some Python code for visualising ASCII image formats such as PGMs (esp. from the Finite Transform Library (FTL)). These formats are commonly used because of their simplicity in implementation.

EDIT: The sMILX Viewer (v1.0 Alpha and above), part of the SMILI project for scientific visualisation now supports these PGMs. Just drag and drop the file into the viewer window to visualise them!

The ASCII PGM format for example has a simple header made up of a format string (made of two characters) such as 'P2' on the first line, followed by a comment line, the dimensions of the image (width and height) and the bit depth. Finally the data is next. An example is given below:

P2

# Generated PGM.

101 101

255

184 180 188 199 202 203 200 195 195 202 198 186 181 156 ........

Since most PGM and image viewers always expect a bit depth of 8-bit, the result isn't always shown correctly. I have written a simple Python module to do this type of reading and also utilise 32-bit PNGs as well. The full module can be found in this gist. The section for PGMs is as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

def readPGM(name): ''' Read a PGM image, where the PGM format is that of the FTL library for NTTs and FRTs. This is a slightly modified format where values are not limited to 8-bit values. Returns array of image and bit depth (image, depth) ''' inFile = open(name,"r") #read header formatLine = inFile.readline() commentLine = inFile.readline() if not "P2" in formatLine: print "Error: PGM not in correct format (P2)" print "Comment:", commentLine width, height = [int(x) for x in inFile.readline().split()] # read dimensions print "PGM Size:", width, "x", height bitDepth = [int(x) for x in inFile.readline().split()] # read bit Depth imageList = [] for line in inFile: # read remaining lines valueList = [int(x) for x in line.split()] #read integers on each line for value in valueList: imageList.append(value) #append as 1D list # print imageList #store as array image = np.array(imageList).reshape(height, width) return image, bitDepth

Then to load results from FTL, simply use a script as:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

# -*- coding: utf-8 -*- """ Test Read PGM member Created on Sun Jan 25 21:52:38 2015 @author: shakes """ import imageio image, depth = imageio.readPGM("lena.pgm") frtSpace, depth = imageio.readPGM("frtSpace.pgm") #Plot import matplotlib.pyplot as plt #plot images fig, ax = plt.subplots(nrows=1, ncols=2, figsize=(16, 5)) plt.gray() ax[0].imshow(image) ax[0].axis('off') ax[0].set_title('Image') ax[1].imshow(frtSpace) ax[1].axis('off') ax[1].set_title('FRT of Lena') plt.show()

The result is a Matplotlib plot of the images, both of Lena and her FRT space:

You can adapt it for your image format by just changing what/how the header is read. Feel free to fork the gist as I will put the latest version there. This module is part of a pure Python version of FTL coming soon, so watch this space..... better yet... Follow!

PS: A FTL plugin for my scientific visualisation software SMILI is on the way too. More soon in the next version.

UPDATE: Scientific visualisation (open-source) software SMILI supports 32-bit PGMs natively now via the FTL plugin. Get the binaries from GitHub or Sourceforge.

Cheers Shakes - L3mming

SMILI: A simple open-source framework for scientific visualisation

I'm pleased to announce the release of SMILI, a simple framework for scientific visualisation, under a BSD license on SourceForge and GitHub.

In this post, I thought I would cover some of the cool features and command-line utilities available for researchers and developers. The overall documentation can be found here. This post will hopefully be complimentary to this and the videos of SMILI already available on YouTube.

Some of the additional features of SMILI are the ability to share the same processing and display capabilities of the GUI applications with their command-line counterparts. For example, you have the following command-line tools to assist you in your research:

1. milxImageViewer and milxModelViewer - These are fast, no-nonsense viewers for n-D images and 3D polygonal data, such as triangulated surfaces. They don't load pesky plugins, they just allow you to view your data fast. The right click options are still available, especially the processing elements. However, any interaction between windows is no longer possible for obvious reasons. You can use sMILX for this.
2. milxOverlay and milxAnimate - These applications allow you to take screenshots/movies of models and images together with pre-defined views (which can be created in sMILX). This is great for visually inspecting your results or for websites when there are a lot of them. There is a batch script in 'scripts/' directory to batch these applications over multiple threads for quick generation.
3. milxImageApp and milxModelApp - These are the 'swiss-army knife' like applications for images and models respectively. With them you can simply process all the images/models you provide via the command-line in the same way with many of the algorithms available in sMILX. For example, to threshold labelled images (having Nifti format) within a certain range and storing result in 'auto', we could simply:
   milxImageApp --threshold --above 225 --below 180 *.nii.gz -p auto/auto_
4. milxLabelVisualisation - Sometimes it is necessary to visualise labelled images using iso-surfacing or marching cubes or volume rendering with certain colour maps. This application provides a way of doing these things with off-screen rendering.
5. milxAssistant - This application provides a simple web browser interface to explore the SMILI documentation in a similar fashion to Qt Assistant.

As updates occur, I will post notices on twitter and Google+. Major development updates will be posted here in this blog. Currently, a journal publication for SMILI is under review entitled:

"Focused Shape Visualisation via the Simple Medical Imaging Library Interface"
Visualisation and Computer Graphics, IEEE Transactions on, 2014, Submitted

Upon acceptance, more details, plugins and revision history will be released. If you have any feedback, feel free to post as a ticket, email the mailing-list or message me on SourceForge.

Cheers Shakes - L3mming

Robust Digital Image Reconstruction Example

In this post, we discuss how to employ the digital image reconstruction technique of Chandra et al. (2014):

Robust digital image reconstruction via the discrete Fourier slice theorem

S Chandra, N Normand, A Kingston, JP Guedon, I Svalbe

IEEE Sig. Proc. Lett. (2014)

using the FTL (implemented in C, available via LGPL license).

This method takes a sufficient set of discrete (rational angle) projections assuming the Dirac pixel model, i.e. digital image sampling where lines have said to have sampled a pixel iff the line goes through the centre of the pixel, and reconstructs them in O(nlogn), where n=N^2. Sufficiency is classified as those projections meeting the Katz criterion, i.e. basically all bins are sampled at least once and no unambiguous  solution (i.e. a ghost) cannot fit within the image. See also:

Fast Mojette transform for discrete tomography

SS Chandra, N Normand, A Kingston, J Guédon, I Svalbe

arXiv preprint arXiv:1006.1965

Once you have FTL built, you should have four binaries for this method:

• fmt_angles - Select the angle type and generate the rational angles for given n and N, the image and reconstruction (FFT space) sizes respectively.
• mt - Compute the discrete (rational angle) projections of the image, also known as the Mojette transform (MT).
• mt2frt - Convert the projections to those of the FRT/DRT, which are the inverse FFTs of the slices of the 2D FFT.
• ifrt - To reconstruct the resulting FRT projections in O(nlogn), no relation to the n before.

To illustrate the process we given a tutorial here of the whole process.

1. First, we crop Lena image to a 128x128 image of Lena from the centre:

./crop lena512.pgm 128 128 0 lena128.pgm

2. We create the angle set, we choose the L1 minimal set since it has a nice symmetry:

./fmt_angles 128 256 1 mt_angles_128_in_256.txt

3. Next we compute the MT:

./mt lena128.pgm mt_angles_128_in_256.txt mt_lena128.pgm

    Note that if you already have projections, such as those of a sinogram, then see this Google Groups     discussion. You can find the publication by my colleague Andrew Kingston on how to do this here.
4. Convert the MT projections into FRT space

 ./mt2frt mt_lena128.pgm mt_angles_128_in_256.txt 128 256 1.0 frt_lena128.pgm

5. Invert the FRT projections in O(nlogn) using the discrete Fourier slice theorem

./ifrt frt_lena128.pgm recon_lena128.pgm

This gives our nxn result reconstructed and padded into the NxN space.

EDIT: See my other post about visualising these and FTL (or PGM files) results in general.

HTH
Cheers Shakes - L3mming

Australian Dell XPS 13 Touch (9333) 2014 with Ubuntu/Linux Experiences

After many hours trying of trying to get my old laptop battery life above 3 hrs, I finally bit the bullet and bought a new laptop.

I eventually went for the Dell XPS 13 Touch (i7, 256 GB SSD, 8 GB RAM) and I'd thought I should put down a few words about my experiences for anyone looking for a good portable developer laptop.

My findings in summary: a damn fine developer laptop! Great backlit keyboard, Ubuntu works (details below), fantastic battery life, quiet and portable to the max.

Basically, weights as much as a 10" tablet, about the same size despite having a 13" Full HD touch screen. First day I took it to work and on battery compiled ITK within 10 minutes (as well as some custom libraries), installed a ton of stuff from the repositories, browsed the web, did some development and remote login work and still only consumed only two thirds the battery at the end of the day. My colleagues also loved the styling and were impressed overall.

Why not a Mac Book Pro?

I have to say the Mac was tempting. It was a little cheaper (lower spec though), probably more robust (by a little) and more battery life (only just) and Unix style OS with native MS Office. In the end, and after having used a Mac for a few weeks now for deploying some software, it became a matter of personal belief of what software should be about: FLOSS.

For example: It seems to customise and to get 'non-standard' elements working requires buying stuff. Want to write to NTFS filesystems? Buy software. Want to customise dock? Buy app. Even the open-source and free SciTE is available for purchase on the app store. Sigh. This together with no middle mouse click, no uninstall for installers and deployment issues for 10.6 and 10.9 push me over the edge and I don't regret one bit (so far... hehehe).

Installing Ubuntu

Installing Ubuntu was almost seamless. As per usual, follow all the backup routines etc. before installing. A few minor hurdles were encountered:

• Kernel 3.14 is required to get the best battery life for i7's, thus I used distrowatch.com search page to work out the distros that had the latest linux kernel. This was OpenSUSE factory and Ubuntu Snapshot
• I used the Ubuntu snapshot (06/08/2014) and the iso to usb installer that comes with Linux Mint (which was installed on my desktop) to put it on a USB stick since the XPS 13 has no DVD-drive.
• Make sure to shrink the Windows partition in Windows as during the partitioning stage, it kept trying to install Ubuntu to the USB stick since it was a large (16 GB) stick and only disk with space, making it difficult to resize in the installer itself. With all the EFI stuff, it better to let the installer set it up for you. OpenSUSE installer did the same thing too.
• After the installer finished and I rebooted, everything worked! Even the touch screen. I was shocked lol. Not usually my experience in the last 10 years with Linux of any flavour. Linux has come a long way in the last few years! Even Ubuntu looked fabulous and it was good to hear the drum sound again after being away from it for so long. ;D
• When I say everything, I kind of exaggerated a little. the Wifi didn't because of propriety firmware that just needs linux-firmware-nonfree package installed.
• The touch device still has a bug and is not recognised properly, though it works fine. I got middle mouse click working with three finger tap (disabled on Ubuntu by default.... wtf).

Hope this helps someone.

Cheers Shakes - L3mming

Linux Distros for Laptops and Power Saving Tweaks for a 2nd Gen i7 Machine

In this post I will note down my experiences in finding a suitable Linux distro for my HP Pavilion dv6 and some power-saving or improving battery life tweaks I now use.

This laptop is really a portable desktop, that is a burn-top rather than a laptop. It is heavy (~2.5 kg), power hungry but very powerful, when it is not overheating and burning your lap.

The first distro I tried was OpenSUSE 12.3 with KDE. This worked out of the box with everything and was a joy to use when AC is plugged in. However, the battery life wasn't great (~1 hour) and it over heated constantly. I upgraded to 13.1, but that process broke my C++ development environment as some libraries became unreachable. Probably fixable but I felt it time to try something else/new.

Battery life and overheating

To improve battery life, I installed TLP and followed this very useful article on howtogeek to tweak this as much as possible.

It is also important to update to the latest video card drivers, AMD/ATI drivers in my case. ATI has settings in Catalyst for power saving and OpenSUSE has improvements for Radeon cards. To get a sense of what is consuming the power, install PowerTop.

To reduce overheating, I underclocked the CPUs to a max of 1.8GHz from 2.0GHz on AC power, to 1.2 GHz and changed the default governor to 'powersave' when on battery. BTW, this trick also works on windows by changing the max execution from 100% to 90% etc. (see this post). This improved battery life and reduced stuttering from overheating, especially if streaming video.

The result was an increase of battery life from 1hr to 2.5 hrs and almost zero fan noise on OpenSUSE 13.1 KDE. All bets are off when you start compiling code however. ;)

Linux Distros for Laptops

KDE is probably the best desktop manager right now along with Cinnamon, but both are processor hungry etc. and suited to desktops so I started to look into other options.

I read that XFCE and LXDE managers would be better for laptops in various places. I had tried these before a while ago and they were very lightweight and snappy. But they were missing important customisation features and looked too stripped down and ugly a few years ago.

I tried XFCE via XUbuntu 14.04 and it was great and looked stylish. Installed flawlessly, boots fast and just works (WiFi, boot splash, suspend etc.). The search feature in the menu like KDE and Cinnamon is also great, but had issues picking out all the installed apps. My touch pad lost its ability to track three fingers and that, combined with a warning that BIOS detected 'overheating', pushed me to look elsewhere. I would reccomend this for older machines or those lacking CPU power.

Next one I tried was elementaryOS, an Ubuntu based distro which I saw running on a colleague's machine. It looked stunning, snappy and workable. After installation, I got the same BIOS error (not surprising since the stable version at the time was based on Ubuntu 12.04) and the panel was very difficult to customise. It looked so good I wanted to stay but even after an hour of searching on the web I couldn't get a working CPU monitor or weather indicators. A distro I would setup for the Mrs or family as it was clean and elegant.

In the end, I ended up re-installing XFCE version of OpenSUSE 13.1 and haven't looked back (so far). The Ubuntu version of XFCE looks better, but the BIOS error disappeared and the track pad worked again. Combined with Cairo-Dock, the final result is reasonable looking and battery life slightly better than the KDE version. I simply switch between KDE and XFCE depending if on battery or not.

OpenSUSE with XFCE and Cairo-Dock

Hope this is useful to someone.

Todo: I need to try the XFCE version of Linux Mint and Lubuntu.

Cheers Shakes - L3mming

Getting Android CMake to work....

I recently wanted to get one of my libraries - FTL to build and run on my nexus 7 tablet. Since everything is done via CMake, the obvious question was could I just get the Android NDK toolchain setup and building without any changes to my library? (For those looking to setup it up manually without CMake see this post)

First step.... getting Android CMake to work

There are a number of things spread around the net but nothing conclusive to point out obvious noobie mistakes I came across.... so I documented my findings just in case someone else has the same issues:

1. After setting up the NDK r9 x64 version as:

export NDK=~/Dev/android-ndk-r9

$NDK/build/tools/make-standalone-toolchain.sh --platform=android-5 --install-dir=$HOME/Dev/Install/android-toolchain --toolchain=arm-linux-androideabi-4.8

It is very important to put the whole path in and not the './' instead. Otherwise it will complain about the NDK install not being found. The toolchain should match your device. [From Android CMake docs]

2. Next you need Android CMake. The project on Google Code is no longer supported, but the OpenCV version appears to be the latest version, so grab it from here. You just need the .cmake file, so I grabbed the Google Code version and replaced the android.toolchain.cmake file with the latest version. I placed this at ~/Dev/Install/android-cmake.

3. Test it on the Android CMake samples available on the Google Code repository. Setup the environment:

export ANDROID_NDK=~/Dev/android-ndk-r9
export ANDROID_NDK_STANDALONE_TOOLCHAIN=~/Dev/Install/android-toolchain
export ANDROID_NDK_TOOLCHAINS_PATH=$ANDROID_NDK
export ANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.8
export PATH=$ANDROID_NDK_STANDALONE_TOOLCHAIN/bin:$PATH
export ANDROID_CMAKE=~/Dev/Install/android-cmake
export ANDTOOLCHAIN=$ANDROID_CMAKE/toolchain/android.toolchain.cmake
alias android-cmake='ccmake -DCMAKE_TOOLCHAIN_FILE=$ANDTOOLCHAIN -DANDROID_NDK=$ANDROID_NDK -DCMAKE_C_COMPILER=arm-linux-androideabi-gcc -DCMAKE_CXX_COMPILER=arm-linux-androideabi-g++ -DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.8 -DANDROID_NDK_TOOLCHAINS_PATH=~/Dev/android-ndk-r9'

The last 'alias' command should be on one line or break it up slashes. The tricky bit is the distinction between the ANDROID_NDK and the ANDROID_NDK_STANDALONE_TOOLCHAIN. The latter is good for directly accessing the compilers etc. and the former is needed by Android CMake to find the NDK. If not set correctly, you get the "Could not find any working toolchain in the NDK. Probably your Android NDK is broken" message.

If all goes well then doing the following will work:

mkdir build
cd build
android-cmake ..
make -j 4

At 'android-cmake' line, I used the Curses front-end for CMake. Replace it with your preferred front-end. The rest of the documentation for Hello-CMake example should work fine. More on Android CMake porting of libraries later....

Cheers Shakes - L3mming

Slices, Stacks and ITK

I recently had to program some registration and decided to use ITK. However, there are a number of things I could not find well documented, so I thought I'd put stuff I had to figure out in this post, in the hope it will be useful to others.

Firstly to load a TIFF series or image stack, which is well documented in the ITK Software Guide (Section 7.11), the code looks like this:
 
typedef unsigned short PixelType;
typedef itk::Image< PixelType, 3 > ImageStackType;
typedef itk::ImageSeriesReader< ImageStackType > ReaderType;
typedef itk::NumericSeriesFileNames NameGeneratorType;

///Generate Numerical File Names
NameGeneratorType::Pointer nameGenerator = NameGeneratorType::New();
    nameGenerator->SetSeriesFormat( argv[1] );
    nameGenerator->SetStartIndex( first );
    nameGenerator->SetEndIndex( last );
    nameGenerator->SetIncrementIndex( 1 );

///Read Stack as TIFFs
ReaderType::Pointer stackReader = ReaderType::New();
    stackReader->SetImageIO( itk::TIFFImageIO::New() );
    stackReader->SetFileNames( nameGenerator->GetFileNames() );
    stackReader->Update();

Here argv[1] will be an sprintf format string like 'file%03d.tif' for reading files named file001.tif, file002.tif etc. The typedefs may appear to be overly used, but results in much leaner and reusable code.

To view the stack, use the ImageToVTKFilter (found following these intructions) like:

typedef itk::ImageToVTKImageFilter< ImageStackType >  StackConnectorType;

///Export to VTK
StackConnectorType::Pointer stackConnector = StackConnectorType::New();
    stackConnector->SetInput( imageStackFiltered );
    stackConnector->Update();
    stackConnector->GetOutput()->GetExtent(bounds);
    cout << "Size: " << bounds[1] << "x" << bounds[3] << endl;

///Display
DGVImageVTK *imageStackView = new DGVImageVTK;
    imageStackView->alignImages(false); //Has no effect?
    imageStackView->setName("Image Stack");
    imageStackView->SetInput(stackConnector->GetOutput());
    imageStackView->generateImage();
    imageStackView->show();

The result of the filter is passed to my DGV Image Class which wraps VTK. You can find DGV here. You should be able to pass the connector to vtkImageViewer2 class also.

There are two main posts/email-list entries that are useful for slice traversal and processing. The first is the method using itk::PasteImageFilter. I found this to work but contained large amounts of code and was very slow.

The second method was the JoinSeriesImageFilter, which a lot better but missed some newbie ITK stuff, which I missed and resulted in all slices of the result containing the same slice. The end working result is code that looks like:

///Iterators
typedef itk::ImageSliceConstIteratorWithIndex< ImageStackType > SliceConstIteratorType;

///Extractors

typedef itk::ExtractImageFilter< ImageStackType, ImageType > ExtractFilterType;
typedef itk::JoinSeriesImageFilter< ImageType, ImageStackType > JoinSeriesFilterType;

///Traverse through slices and produce new output stack
///Setup Slice Iterators which will iterate through slices in the stack
SliceConstIteratorType inIterator( imageStackFiltered, imageStackFiltered->GetLargestPossibleRegion() );
    inIterator.SetFirstDirection( 0 ); ///x axis
    inIterator.SetSecondDirection( 1 ); ///y axis

///Setup Image Stack that will be Joined together
JoinSeriesFilterType::Pointer joinSeries = JoinSeriesFilterType::New();
    joinSeries->SetOrigin( imageStackFiltered->GetOrigin()[2] );
    joinSeries->SetSpacing( imageStackFiltered->GetSpacing()[2] );

for(inIterator.GoToBegin(); !inIterator.IsAtEnd(); inIterator.NextSlice())
{
    //cout << inIterator.GetIndex() << endl;

    ///Setup region of the slice to extract
    ImageStackType::IndexType sliceIndex = inIterator.GetIndex();
    ExtractFilterType::InputImageRegionType::SizeType sliceSize = inIterator.GetRegion().GetSize();
        sliceSize[2] = 0;
    ExtractFilterType::InputImageRegionType sliceRegion = inIterator.GetRegion();
        sliceRegion.SetSize( sliceSize );
        sliceRegion.SetIndex( sliceIndex );

    ///Pull out slice
    ExtractFilterType::Pointer inExtractor = ExtractFilterType::New(); ///Must be within loop so that smart pointer is unique
        inExtractor->SetInput( imageStackFiltered );
        inExtractor->SetExtractionRegion( sliceRegion );
        inExtractor->Update();

    ///Operate on Slice
    InvertorType::Pointer invertor2 = InvertorType::New(); ///Must be within loop so that smart pointer is unique
        invertor2->SetInput( inExtractor->GetOutput() );
        invertor2->Update();

    ///Save Slice
    joinSeries->PushBackInput( invertor2->GetOutput() );
}

///----------
///Write out multi-page TIFF of the result
joinSeries->Update();
WriterType::Pointer writer = WriterType::New();
    writer->SetFileName( "registered_stack.tif" );
    writer->SetInput( joinSeries->GetOutput() );

try
{
    writer->Update();
}
catch( itk::ExceptionObject & err )
{
    std::cerr << "Write Output Exception caught !" << std::endl;
    std::cerr << err << std::endl;
    return EXIT_FAILURE;
}

The iterator goes through all slices without needing to know anything about dimensions or sizes. You will have to tell the ExtractFilter the slice to extract, this is described using ImageRegions.

The operation applied to the slices is simply an inversion of the greyscales, especially useful if the slices are negative images. Note that the pointers are declared within the loop, this is important because the pointers remain valid only within the loop and automatically deleted since we are using SmartPointer's. The effect is that ExtractFilter then points correctly to the current slice. The result is written as a multi-page TIFF file, which you can open in ImageJ etc.

The above operation was used as a test, which will be replaced by registration code. If you are applying filters slice by slice, look into the SliceBySliceImageFilter that can be found in the Code/Review branch of ITK atm. To invert greyscales and then rescale intensities on each slice, you get:

typedef itk::InvertIntensityImageFilter< ImageType >  InvertorType;
typedef itk::RescaleIntensityImageFilter< ImageType >  RescaleIntensityType;

typedef itk::SliceBySliceImageFilter< ImageStackType, ImageStackType, InvertorType, RescaleIntensityType > SliceFilterType;

///Filter Stack
InvertorType::Pointer invertor = InvertorType::New(); ///Invert Greyscales
RescaleIntensityType::Pointer rescaler = RescaleIntensityType::New(); ///Normalise image values
    rescaler->SetOutputMinimum( 0 );
    rescaler->SetOutputMaximum( ImageMax );
    rescaler->SetInput( invertor->GetOutput() );

///Apply the filters to each slice of the stack
SliceFilterType::Pointer sliceFilter = SliceFilterType::New();
    sliceFilter->SetInput( 0, imageStack );
    sliceFilter->SetInput( 1, imageStack );
    sliceFilter->SetInputFilter( invertor );
    sliceFilter->SetOutputFilter( rescaler );
    sliceFilter->Update();

Thats it for the moment. More on registration later and code release later.

Hope that helps.
Cheers Shakes - L3mming

Packaging Qt Applications for Ubuntu/Debian

I have recently attempted to get my Discrete Geometry 3D Viewer capable of building Debian packages, so that one may install DGV without needing to compile it or worry about dependencies for other Ubuntu Distros (like the question I got from a potential user).

I believed there would be a substantial discussion on how to do this for Qt applications, but I could only find the Maemo Guide useful. It gets even more difficult if you want to do multiple binaries from a single source. Hence, I have documented my findings of this topic in this blog.

Useful Links that I used:
Complete Ubuntu Packaging Guide
Qt App Maemo Guide
GPG Guide
Pbuilder Howto
Using Local Packages

Firstly, my situation is the following. I have three side-by-side dynamic libraries, dgv-base, dgv-contrib and dgv-vtk. The libraries are divided based on dependencies of Qt, None, and dgv-base & Qt & VTK respectively by design. Then there's the actual DGV application which depend on these libraries.

First the multiple libraries. Begin by renaming the source directory, with the name of the package and the version (with a dash as the separator, this is important). For example

libdgv-0.15

Assuming that your source is in the state you wish to distribute it, create the tarballs so that you have two tarballs

libdgv-0.15.tar.gz
libdgv_0.15.orig.tar.gz

Note the underscore between the library name and the version. This is very important.

Change to the directory of the source and execute

dh_make -e your.maintainer@address -c GPL

where the "-c GPL" assumes you are using GPLv3 license and "your.maintainer@address" is your email address. This will ask you a series of questions, where you should select library if you're doing a library or a single binary if you're doing a binary. I will assume a library for the aforementioned reasons.

This step will create a directory called "debian" with all the Debian package configuration files. Remove the example files, they are not needed for what we are doing as far as I know.

rm *.ex *.EX

Please read these pages of the packaging guide for the remaining files. You need to edit the control and rules files as I have for DGV (also see Maemo guide for a simple single binary example). Things to watch out for is also given in this link (at the end). Fill the changelog and copyright files as described by the package guide, making sure to use the name of the overall package where applicable and to match the author email to your GPG name. If you do not have one, you need to create one using the GPG guide to sign your packages.

For multiple binaries, there is one last step. You need to create the ".install" and ".dir" file for each package. The former has the list of files to be installed, but in a wild card format (e.g. "usr/lib/lib*.so"). The latter is where the files are to be installed (e.g. "usr/lib").Note that there is no "/" in front as per normal Linux directories from root. This is because they will be relative paths and when the real package is built, it will build it from the root. See the debian files I created for DGV here.

Now to build your package, we will use Personal Builder (pbuilder), a way to build your package using only the minimal (initial) Ubuntu base/setup plus your dependencies. This is the best part that makes constructing packages most useful. If your dependencies are correct, then the user just installs the package and Synaptic or apt-get just installs everything you need to get it going. Once a pbuilder environment is setup (which can be made to suit for building on different architectures and Ubuntu distros), the package is built with your configuration and placed into

/var/cache/pbuilder/result

Only downside is that all the base packages will be downloaded from the Ubuntu repository, which could take a while. If you can't use the Internet for whatever reason, you can still build it the packages using

debuild

when within the source directory. You might also want to do this initially to ensure that its all working correctly.

First create the pbuilder environment by

sudo pbuilder create --distribution $(lsb_release -cs) \
        --othermirror "deb http://archive.ubuntu.com/ubuntu $(lsb_release -cs) main restricted universe multiverse"

Then build the descriptor file

debuild -S

If you did a "debuild" already, then this step is unnecessary. Once the descriptor file is present, then build using pbuilder

sudo pbuilder build *.dsc

To build as a different distro or architecture, see this. Hopefully, all works fine and you get a few packages. There might be a few warning from Lintian. Ensure that the descriptions lines are not too long or google the lintian warning. If your warning says "empty-binary-package" then your files are installed in the wrong places. For Qt projections, I install the files of the library into "debian/tmp", then move it out into the relevant package directories by using "dh_movefiles -p$@ -Xcontrib -Xvtk usr/lib" command in the rules (see the wiki). The "-Xitem" tell the movefiles app to ignore the files with the string "item" from the file names.

Finally, on a Live CD or clean install, check you packages by installing them. Pbuilder didn't pickup the fact that I had incorrectly named libvtk5.2 as libvtk5 for one of my packages, so I recommend it.

I then build the dgv application and its all done. Hope this helps. I will post up more on how to use local packages later when I have it working.

Cheers Shakes - L3mming