How to Install Jupyter and IPython from Source using Cmake
This is something I did not think would work. But the book I bought had all the examples running in Jupyter Notebooks on IPython - the interactive Python. And I now have it working, and it is good.
I have multiple machines in the lab, running an earlier version of Fedora Linux. The kernel has been custom-compiled - and uname reports it as: "Linux 2.6.25-14.fc9.i686#1 SMP". These boxes are crazy stable, and are used to do real work. But I have a couple of ACER laptops also running this Linux OS, and decided to try to get one working with modern Jupyter and IPython so some of my data-analysis work could be done using modern methods, instead of APL and Xerion (environments that are 20+ years old). Using modern code will give me access to the TensorFlow AI development tools, which look pretty cool, as they are the next generation of back-propagating neural networks - a logical outgrowth of the Xerion toolsets, which are Tcl/Tk and C based.
First, I had to
get a modern (or more or less modern) Firefox running. That involved installing and compiling GTK2 and GLIB from source, which was amusing. Firefox version 34 was downloaded from Mozilla as a tar.gz tarball. Used "gunzip" to remove the .gz
and then standard "tar -xvf" to get the files in a directory. They are binaries, with libraries and various modules, omni.ja (a giant java blob, I think), and a bunch of default components and profile stuff. Wouldn't run - kept generating
error: "libxul.so: undefined symbol:
gtk_widget_set_can_focus". This error results from using the wrong library files for GTK and GLIB. It took me days to resolve this - I only kept at it because I had another laptop with the
same OS, and had the Firefox 34 running perfectly. It's an easy fix, as FIrefox's runner thing (used to be called "xulrunner") was just pointing to the wrong library.
The problem was resolved by figuring out that when I built the new GTK and GLIB stuff from source, the library files were put it "/usr/local/lib", not in the main system area, "/usr/lib". This took *days* to determine - I had done "rpm -qa --info <packagename>" on most of my gtk related packages by the time I figured this out. The typical "make install" puts the generated dynamically loaded libs into the /usr/local/lib to avoid trashing system operation, which is probably relying on the stuff in /usr/lib. There is method to this madness. All that was needed to get Firefox to work, was to have it point at the right dynamic libs when it started. If you put your tarball for Firefox in "/home/Firefox34", for example and tar -xvf it there, you will get a "firefox" subdir. You can create at firefox invokation script with two lines: (put it somewhere it can be found...)
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
/home/firefox34/firefox/firefox
and that will run Firefox ok, as it now will have the newer GTK2 and GLIB stuff it needs. (Here is link to StackOverflow note about LD_LIBRARY_PATH: https://stackoverflow.com/questions/4743233/is-usr-local-lib-searched-for-shared-libraries )
OK, once you get Firefox working (you will need it for Jupyter/IPython Notebooks), then you can build the stuff for IPython.
I had already built and installed Python 2.7.14 - the latest, most recent stable release of the Python 2.7 development stream. I really like this version of Python (the 2.7 versus the 3-series stuff), because it will not change and "evolve". Nowadays, stability is vastly more useful than the rapid-change-for-the-sake-of-change that drives development in the universities. Business has to get sh*t done. It just does.
I built "pip" by downloading the pip generator thing as an image, checking its hash values against several different sites. Then, I got pip to run on the Python 2.7.14. (Getting Python 2.7.14 running is tricky - you will often already have an older Python (I had 2.6) installed and it will be used to do stuff. I tried to completely overwrite my system Python, but ended up with Python2.7 stuff everywhere (in libs and share dirs...). But running Python at cmd shell gives me the 2.7.14 version, and pip worked. Attempts to just "pip install Jupyter" would fail with undefined stuff.
It turned out I needed to build "libzmq" and "pyzmq" from source. I created a working directory called "tfhack", and put everything in there.
Then, I needed a bunch of other stuff. I had to download and build: cmake, curl, git. To get a nice source clone, you run "git" to clone directly from git to a target directory. I downloaded git-2.1.4.tar.gz, and built a local "git". Of course, I also needed curl and cmake, which I also downloaded and built (using traditional: ./configure, make, make install). I needed git to "git clone" the "libzmq" and "pyzmq" source trees. The install instructions are unhelpful on these source trees, as they still reference the traditional "./configure, make, make install" build procedure, and there are no "configure" files. WTF?? you might ask in frustration? (I certainly did!). Turns out you have to use "Cmake" to do what "configure" used to do, and you have to do it in a temporary (or permenant) subdirectory. After a few rants that made me sound like Adolf in the Bunker, I have to admit that the idea of Cmake is good. Doing a build in the subdir is useful, and the build-pipeline is wise - the main problem with Cmake is that it is just not documented anywhere - including in the source clone from git! Oh my... (I finally found a Cmake tutorial that helped me here: http://preshing.com/20170511/how-to-build-a-cmake-based-project/ Mr. Preshing is a Scholar and a Gentleman for publishing that tutorial.)
I also made some tweaks to some of the .cpp files to address minor stuff. You need to compile libzmq first, then pyzmq. Note: if your Linux distro has libzmq stuff ("ZeroMQ" I think it is called...), then this won't be necessary. Your "pip install Jupyter" command (entered at command shell), should just work. Mine, of course, did not.
TO compile the cloned "libzmq" (oh, the git clone url address is: https://github.com/zeromq/libzmq ), you need to do this, essentially:
mkdir cmake-make
cd cmake-make
cmake .. -G "Unix Makefiles" (edit: I had left out the ".." here, which to parent dir.)
Then, you can run "make" and then "make install" as per usual, and you should get a working lib in your /usr/local/lib directory. After I built libzmq, I did the same process to build pyzmq. (The git clone url for pyzmq is: https://github.com/zeromq/pyzmq ) Check your /etc/ld.so.conf file to make sure you are looking in /usr/local/lib as well as /usr/lib. Again, typically your system-critical libs are in /usr/lib, and your homebuilt hacklibs are in /usr/local/lib. If you just want to use the stuff in /usr/local/lib, then set the LD_LIBRARY_PATH var prior to running your program.
I needed to get the libzmq and pyzmq stuff built to get jupyter to build. After I got the libs built from source, I ran:
pip install pyzmq
And I think that worked. (My notes are a little unclear here..) Main thing, is to try to build Jupyter. I just did:
pip install jupyter
And that (to my great astonishment) actually worked. I had Firefox running, and from a command shell, I just entered:
jupyter notebook
And suddenly, I am looking at a web-page showing a directory listing of my "tfhack" directory! (See the small picture at right of this article)...
My setup is still a bit kludgy, as I have to start Firefox first, and then run "jupyter notebook", but the local webserver starts, and I correctly see: "localhost:8888/tree" as the URL in the browser URL topmenu.
To get an interactive session, where you can just enter Python commands, and run stuff and see graphics and such, you either load up an old notebook file you already created (Jupyter/IPython Notebook files have the form <filename>.ipynb), or you press the "New" button, and you will get the familiar interactive session where you can enter some Python commands.
This environment is VERY COOL. We are finally (FINALLY!) back to something useful, that is not just an Excel spreadsheet. I am pretty sure this paradigm might just catch on. Having interactive computing (like APL, for example), was there at the very beginning - and it is nice to see it make a return. Using this type of environment is how people learn. It is important that this kind of approach be available, as it lets kids learn fast, learn well, and have fun doing it as they play with the technology.
Below, is a cool thing. I used the np.random.randn function to create 100 million random numbers, drawn from a standard normal distribution with mean 0 and variance of 1, and then multiplied them by 20. Then, the matplotlib is used to create a histogram with 25 bins between -50 and 50, and is shown. This entire program took only 15 seconds to run on my dual processor, 2.0 ghz Intel-powered Linux ACER Travelmate laptop.
Here is the link to the documentation on numpy function "numpy.random.randn" function.
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.random.randn.html
Wow. 100 million random numbers, drawn from a normal distribution with mean 0 and variance of 1, multiplied by 20, and then shown in a histogram with 25 bins. This program ran in only 15 seconds on my Linux laptop (an ACER Travelmate with Centrino Dual processor running at 2.0 Ghz). We can now simulate anything, it looks like. "Hydrosims", anyone?
This is the image test suite, showing the original numeric generated "testimg.png" file, and the gray-scale version, displayed using Tck/Tk and matplotlib, with scikit-image in Python 2.7.14, running on a Linux laptop. It provides same results as the CentOS Linux, and the Windows Python 2.7.12 version, also running same suite of libraries and test programs. For cross-platform development of data-research applications, Python works well. It's ability to run all the current software on an older Linux platform is impressive, and is the result of building the software from source. I will include a texfile which is basically a running-log of the of the activity and actions I took to get this all working.
Real-time Log of Getting Python, Scikit-Image and Jupyter to run on LInux
Build Notes for Python 2.7.14 on L2-ACERTM - Dec. 3-7, 2017
----------------------------------------------------------------------------------------------------------------
Objective here is to build the same Jupyter/IPython
Data-research environment
on the L2-ACERTM laptop as the L2-ACERT2 laptop, which now runs it all.
This work is done on L2-ACERTM, using source material datasets from
L2-ACERT2. (These are two test-bed laptops, running Linux, which
I have been
using for several years now...)
First, need to get some utilities...
cmake
curl
git
This is done at raw term shell mode, using a bash shell. You
can
do it inside a gnome window and just use term shell, if you want.
I wanted to see if all could be built without starting Xwindows.
cmake
=====
- From root, create the /home/cmake directory
- copy
from L2-ACERT2 /home/cmake/cmake-2.8.12.tar.gz to /home/cmake on L2-ACERTM
- run "tar -xvf cmake-2.8.12.tar.gz" to get the installable source
- cd to /home/cmake/cmake-2.8.12 directory that was created by the tar -xvf cmd..
- run: ./configure to bootstrap cmake configure
- after successful ./configure, run: gmake
- if successful, run: make install
- log into a different userid,
and run: cmake --version
and confirm you see: cmake version 2.8.12
hint: you login to a different session with: Alt-F# (where # is 1, 2 or 3 ...)
I typically login to root on F1, and
then Alt-F2 is my standard non-root userid.
- Now, heres the thing: I have a subdir in the cmake dir called: extra-cmake-modules
and I don't see how I got it there... Hmmm. I also have cmake-2.8.3.tar.gz.
Now,
I did a tar -xvf on the cmake-2.8.3.tar.gz, but it only created cmake-2.8.3 dir,
which I did not use for L2-ACERTM. I don't actually know where I got the
"extra-cmake-modules"
from (called "ECM", apparently for KDE stuff... Maybe Krita
work put them there?) hmmm. Not sure if they are needed.
- I moved the directory from L2-ACERT2 to L2-ACERTM using this command (which does
the move correctly - just replicating the directory. I cd'd to the directory
L2-ACERTM:/home/cmake
when I run this:
scp -r root@L2-ACERT2:/home/cmake/extra-cmake-modules/ .
The above command creates directory /home/cmake/extra-cmake-modules on L2-ACERTM
in the /home/cmake directory. Not sure if I need it.
But I think it was run on
the L2-ACERT2 machine (where Jupyter/IPython is running), as a "build" directory
is in /home/cmake that has the ECM stuff in it. I have absolutely no idea what
I did to compile the "extra-cmake-modules", but I seem to recall they were needed.
From "askubuntu.com" I found this, and did it for L2-ACERTM, as root.
Note that you need to have built "cmake" for this
to work...
cd /home/cmake/extra-cmake-modules
mkdir build
cd build
cmake ..
make
make install
curl
====
- create: /home/curl and cd to it.
- copy from L2-ACERT2 /home/curl/curl-7.20.0.tar.gz
- untar and create source directory with: tar -xvf curl-7.20.0.tar.gz
- this is a traditional procedure, as root.
./configure
make
make install
- confirm with "curl --version". Make sure you have the https protocol and ssh or SSH showing.
git
===
- create: /home/git
and cd to it.
- copy from L2-ACERT2 /home/git/git-2.1.4.tar.gz
- untar and create source directory with: tar -xvf git-2.1.4.tar.gz
- this is *also* a traditional build procedure.
./configure
make
make install
- I spent some serious time getting the Tck/Tk gui stuff for git to work in my
Gnome desktop.
And I got it working, but I did not keep notes... <really big sigh...>
- Recommendation: Keep better notes.
- Ah, damn: When running the "make install" step, I get:
install: cannot stat 'lib/tclindex": no such file or directory
make[1]: *** [install] Error 1
make{1]: Leaving diretory '/home/git/git-2.1.4/git-gui'
make: *** [install] Error 2
- This is probably because I have two versions of Tck/Tk installed. I had to install
(downgrade) to Tcl/Tk 7.x series to get Xerion to run. I had Tcl/tk and Wish 8.5
or something like that, and I think this one needs this version.
- "git" has successfully installed, and is available in command-line mode, but the
GUI will not work, which is annoying.
-
Ah, here is the explanation and fix; On my L2-ACERxx machines, the default Tcl/Tk is
7.3 series, to let Xerion work. But I have apps written in Tcl/Tk and they run using
a more modern Tcl/Tk, version
8.5
- To check the Tcl/Tk version. Type "tclsh" at cmd shell. You get a "%" prompt from Tcl.
info tclversion
7.3
- If I enter "tchsh8.5", and then "info tclversion", I get "8.5". This is key.
So, just hack the GIT-GUI-VARS thing in git subdirectory "git-gui" to point to the
right Tck/Tk version, and your GUI
thing should work for "git", which is actually,
pretty freakin' cool, cause "git" is a genius thing. Just vi GIT-GUI-VARS (it is
just one single line) to change:
Edit (somehow) the file: /home/git/git-2.1.4/git-gui/GIT-GUI-VARS
(and change)
TCL-PATH='tclsh'
to
TCL-PATH='tclsh8.5'
Actually, ***this is not it***. You need to change tclindex somewhere. If I change
==============
the GIT-GUI-VARS and then "make install" or even ./configure, make, make install,
it just changes it back to "tclsh", and complains it can't find /lib/tclindex
(and I need it to get from /lib/tcl8.5/tclindex
?)
Solution: (!)
I am lucky. I sent a note to GitHub folks, cause I was getting an "SSH" connect
error when trying to do a git clone of the stuff I needed for
Python...
Here is note!
---------------------------------------------------------------------------------------------
From: Friday Saarelaht (GitHub Staff)<support@github.com>
To: GemesysCanada
(Mark Langdon) <gemesyscanada@gmail.com>
Hi Mark,
Glad to hear you were able to resolve this issue, and thanks so much for taking the time to let
us know how you went about it!
We'll certainly keep that on file!
Have a wonderful day, and do let us know if there's anything else we can do.
Regards,
Friday
Hi,
I sent a support query re. "git clone" generating a "ssl connect error" with
nns error -12286, and it looks like it was a result of not having all the
various *-devel libraries that git needed.
Specifically, I used yum to update all the -devel versions, and in several
cases, new software
was installed by yum, and I also moved up to the 2.1.4 version
of git. I was able to build successfully from source, and I was able to clone a git
repository from CLI successfully, no errors.
Here is a caveat: I have
two versions of tcl/tk and wish installed - and I had
to edit the git Makefile to point to the more current copy of tclsh and wish, in
order for the lib/tclIndex file to be configured correctly. THis has to happen for
the "make install" step
to work.
And I used "yum install" to update: curl-devel, expat-devel, gettext-devel,
openssl-devel, perl-devel, and zlib-devel.
That let git build successfully on this old Fedora kernel, and the git program
to work against
the github site correctly.
The info provide on the git site "Installing from Source" was most
helpful: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
Thanx for offering this support
facility. Sometimes, just carefully writing
a note, and organizing one's thinking, is enough to point the way to a solution.
ML
---------------------------------------------------------------------------------------------
<I included the note, in case I need to get one of the "-devel" things that I
got via yum. Note: Still don't have dnf working... >
Ok, so the solution:
After you
run: ./configure
Then: Edit the: "Makefile"
and change:
TCL_PATH = tclsh
TCLTK_PATH = wish
to:
TCL_PATH = tclsh8.5
TCLTK_PATH = wish8.5
And then re-run the "make install", and you get a clean install, and can run git
as a GUI in Gnome. Just create a "Launcher" on your desktop, and put this in
the command string:
tclsh8.5 /home/git/git-2.1.4/git-gui/git-gui
And you will get a nice little Git Graphic User Interface
window, where you can
indicate where you get three options: Create a New Repository, Clone Existing
Repository or Open Existing Repository.
Ok, so that has got "git" working, which I need to
clone the stuff for Python.
Oh, I need to get Python 2.7.14.. Arrrgh...
Python 2.7.14
=============
cd /home and as root:
mkdir Python27
scp root@L2-ACERT2:/home/Python27/Python-2.7.14.tar.xz .
Ok, so now, we have Python 2.7.14, but it is in some crazy annoying stupid .xz
format that I cannot uncompress. FFS. Really.
What I did, was to copy
the Python2.7 file to my CentOS box which has a newer
version of tar, and uncompress the thing there. I think that is what I did.
I might have decompressed it on a Windows box or on the Macbook - I can't remember.
SO, just for completeness - I downloaded the xz utils and tried to make a version
of unxz. And that didn't ****ing work, of course, since I have version .17 of gettext,
and the autogen.sh that makes the configure for unxz says
unxz requires gettext .19.
FFS.
SO, assuming you have only one machine, and need to uncompress this dumbshit new
compression format (we need more formats like we need more world conflict...)
I have tried
to build it and make "unxz" actually work.
I downloaded "gettext-0.19*" and ran a ./configure, make, make install to build it.
It is apparently some internationalization thing (i18n?), to report stuff in any
language. Silly. Everyone should just learn and use English. (I'm joking, right?)
After the "./configure, make, make install" to get gettext 0.19, you have to logout
of root, and login again, to confirm you
are using gettext 0.19.6.
The goddamn autogen.sh for xz failed with a "Autoconf 2.64 required", and
I am running autoconf 2.61. Damn and damn again...
I grabbed autoconf-2.67.tar.gz, copied it to the Linux
laptop in a /home/autoconf
directory, and ran: ./configure, make, make install, in the untarred subdir.
Logged off and on again, and "autoconf --version" reports I have autoconf 2.67.
Again, try to run the xz autogen.sh...
Oh hell! The xz shit now demands autoconf 2.69, and automake 1.12.
I had to go get a more recent autoconf tarball. I got the autoconf 2.69, which turned
out to work ok: You get it at: https://ftp.gnu.org/gnu/autoconf/
and download
the autoconf-2.69.tar.gz, dated: 2012-04-24 23:17, reported size: 1.8M
I had to get a current automake also. Got automake=1.15.1.tar.gz from:
https://ftp.gnu.org/gnu/automake/
and noticed that it is from 2017-06-19, so it is reasonably current...
This finally let the "./autogen.sh" thing generate a "./configure" script in my
/home/unxz/xz
directory. Lets see if it will compile...
NO! THE (*%&*))***(% AUTOGEN.SH THING WILL NOT COMPLETE THE ".configure" file create!
IT FAILS WITH A *^$^(%&
SYNTAX ERROR!!!! FOR F.S. GUYS!!!!
THIS IS AWFUL. CANT A SIMPLE "unxz" UTILITY BE OFFERED THAT JUST WORKS???
WTF?
Damn - all this shite just to decompress a &*)(*()&&
copy of Python2.7 that is
done in .xz rather than .gz. FFS!
So, the *(&*ing "xz" ./configure script crashes with a SYNTAX error on
Line: 7174
`LT_PREREQ(2.2)'
WTF?
Oh hell.. finally on the "https://tukaani.org/xz" page, an important notice that
the bloody GNU Autotools requires the following minimum ****ing versions:
Autoconf 2.64 <-- this is wrong. It actually needs Autoconf 2.69
Automake 1.12
gettext 0.19
libtool
2.2
I'm running Libtool version 1.5.24. Arrrgh.
Again, another ****ing /home/libtool directory, with the source from here:
http://savannah.gnu.org/forum/forum.php?forum_id=8210
(mercifully, they offer it in tar.gz format...)
And we build libtool in: /home/libtool
Run: ./configure, make, make install
Logout and login again (probably runs ldconfig
or something..)
And "libtool --version" reports the new version, which is 2.4.6
Ok, then, back to /home/unxz/xz and try to run the ./autogen.sh
This time, the f**ker ran, and we got a ./configure
file.
as root, in /home/unxz/xz, run ./configure, make, make install
Seemed to work, but when I try to run "unxz" I get a f***ing
fail on the attempt to find the m*****&***ing
liblzma.so.5 library.
Arrrgh!
This is a pain. Really ugly. This ****ing library thing is a common
error all over google-searches. I've ****ing done so many google
searches that the f***ing google-AI's have blocked my searching from
my f(((ing IP#!!! I even answered their godddamn CAPTCHA, and it kept
giving me more after each one I did !
Jezzus,
this the most frustrating thing I've done in years. I finally
just sat and thought about it - and looked in /usr/local/lib
and there was the link file "/usr/local/lib/liblzma.so.5 ->
liblzma.so.5.3.99 "
and you just do a "ldconfig" from term shell, as root. Problem
solved. And now, "xz" and "unxz" work! Arrrgh. But it all works,
on my 32-bit laptop, in my old, stable Linux.
Ok, so , lets see if "unxz" will remove the stupid .xz extension from
the Python 2.7.14 source....
YES!! (it works)
cd /home/Python27
unxz Python-2.7.14.tar.xz
( Or, you can also do this: "xz --decompress -k Python-2.7.14.tar.xz
" )
( the -k option will keep the original .xz file. Otherwise, the file )
( just becomes Python-2.7.14.tar, about 7 times the size of the .xz
version)
ls -l
You should see a file reported as:
71096320 2017-12-03 15:04 Python-2.7.14.tar
Now, make sure you have "gpg" installed, and check the file's hash
using sha512.
gpg --print-md sha512 Python-2.7.14.tar
and here is the sha512 hash for the Python-2.7.14.tar file:
Python-2.7.14.tar: B6012693 CE41A3D2 D86D55B1 8A816734 66E09B26 A6E3D794
DB6EFA0A 92AFCBA3 F58CBFEF 44507200 6BA26B6B 39CEEA4B
72E1A29A C4999A7B 85A683CF 3A407BBB
(I typed the above by hand. I've looked at it 10 times - looks right..)
(I am just showing what I got. I downloaded from the Python website.
)
(If you get something really different, send me an email or contact )
(The Python website. The date on my download was 2017-11-17 for the )
(original file.
)
Right. So, we have a "tar -xvf" 'able file. (My tar was/is version 1.19,
hence all this xv and unxv stuff... I just did not want to overwrite such
a basic and critical system utility... .)
So, lets unpack Python and make it work....
tar -xvf Python-2.7.14.tar
and this will make directory;
Python-2.7.14
and we
can try to build the thing....
./configure
make
---> running the make....
The "make" ran, but failed
to build module: _tkinter
I think that is needed to make graphic stuff work... (tk as in Tcl/Tk?)
But I noticed that it gave me the same "python" binary file,
as I
have on L2-ACERT2, so it is going to be installed, if it passes tests...
make test
This runs for quite a while.
I ran it on CentOS box, and on L2-ACERTM.
It reported "Failed" on L2-ACERTM because it could not find sql-lite,
but it passed on CentOS box. The binary is
the same, so I am going
with it, as the L2-ACERT2 laptop runs "jupyter notebooks" correctly.
make install
And the "make install" ran for
a while....
After completion:
Log into a regular userid...
And in a command shell, type "python" and see what happens...
And, it lives! Success.
Login to a regular user account, and running
just "python", and we see:
Python 2.7.14 (default, Dec. 3, 2017, 17:53:05)
[GCC 4.3.0 20080428 Red Hat 4.3.0-8)] on linux2
Type "help", "copyright", "credits", or "license" for more information
>>> _
And we have this part done. Python 2.7.14 is installed and operational.
-----
Python
- Getting it up and running with Pillow and numpy/scipy
==============================================================
Now, we have to start Python, and get the stuff we need. Much to do here..
We will need numpy and scipy and scikit and mathplotlib and jupyter.
We will need pyzmq and libzmq, and those two will have to be built manually.
Notes on numpy: (Very important stuff..)
The numpy is the old IMSL Fortran
Library. It is solid, good stuff.
It is not shit.
Doc on scipy and numpy:
https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.random.randn.html
Notes: To hide warning
crap from Python (very important)...
From stackoverflow:
https://stackoverflow.com/questions/9031783/hide-all-warnings-in-ipython
---------------------------------------------------------------------------
import warnings
warnings.filterwarnings('ignore')
inside ~/.ipython/profile_default/startup/disable-warnings.py.
I'm leaving this question and answer for the record in case anyone
else comes across the same issue.
Quite often it is useful to see a warning once. This can be set by:
warnings.filterwarnings(action='once')
---------------------------------------------------------------------------
First thing, is to get the "pip" program, the "Python install program".
You download an image
as a .py file, and it will get you Python "pip". With
"pip", you can download and install "numpy" and "scipy" and other stuff and
things (libraries) you will need to make Python do useful work.
Download "get-pip.py" from: https://bootstrap.pypa.io/get-pip.py
and put it in your working directory, whatever that is. Mine was called "tfhack",
since I wanted to get Tensorflow working eventually.
Start Python, and try to run get-pip.py. Use the "execfile("<filename>")" cmd to
do this. (Actually, I did this from the system install directory for Python)
[root@L2-ACERTM
Python27]# python
(Python 2.7.14 starts and prints stuff...)
>>> execfile("get-pip.py")
(You should see this download and install: pip, setuptools, wheel )
(and it should report installed: pip-9.0.1 setuptools-38.2.3 wheel-0.30.0 )
>>>
My first attempt at this fail with a syntax error in Line #43, related to
not having ssl stuff available.
You need the ssl (secure socket layer) stuff
to access things with encryption turned on.
On my Fedora box, I just cheated, and did a "yum install openssl-devel",
to
get the development header files (these are C or C++ program include files that
define things that are needed so ssl stuff can be compiled if needed, I think)
(This is why I included the text of that note I sent to github folks, since
it had a list of *-devel stuff I had to get using "yum", so various things
would work. When you are setting up a Linux box, make sure to include *all*
the "development" stuff, or you probably won't be able to build things
from
source. Most of the development header files and such are included in your
version of Linux. They tell what items are actually in the
dynamic library
files that programs use to do things.)
Once you get pip working, you should be able to install some Python libraries,
and this is where you can start getting the payoff for all this work done.
The "numpy" stuff, (for "numeric library for Python") is the old IMSL Fortran
routines, I have been told. Those IMSL Fortran routines were bloody good code,
well used, fast, and error-free. An ounce of debugged, high-grade running code,
is worth many pounds of feature-laden bloatware, and anyone who has ever
built anything that works and is used by real users, knows the truth of this.
The numpy libs also have array stuff for Python.
Start Python as root, and install Pillow - the upgraded PIL (the old Python
Image Library).
This will let you draw images and read and make JPEGs. Then
load numpy and scipy ("scipy" is "Scientific Python").
You run pip from
command shell, not inside of Python. In Python, you want to
use the "import" command to load the library. *** RUN THESE AS ROOT OR THEY
WILL NOT WORK! ***
[root@L2-ACERTM Python27]# pip install Pillow
(Should report: Successfully installed Pillow-4.3.0 olefile-0.44 )
[root@L2-ACERTM Python27]# pip install numpy
(Should report: Successfully installed numpy-1.13.3, size 12.6 mb )
[root@L2-ACERTM Python27]# pip install scipy
(Should report: Successfully installed scipy-1.0.0, size 40.1 mb )
[root@L2-ACERTM Python27]# pip install matplotlib
(Should report: Successfully installed scipy-1.0.0, size 40.1 mb )
This should let you run some sample Python program that can do things.
Now, when I tried to run a program that used PIL (Pillow) to draw an
image, I get this message:
>>> display: unable to open X server `'.
Which is pretty hilarious -- because - duh - I forgot to start
X!
But I did get the .jpg and the .bmp files created.
So, run: startx (which starts the Gnome+Xwindows desktop, a thing of
real beauty...!)
Use a text editor to create this program:
---------------------------------------------------------------------------------------
#
# --- Make and Show immediately
a simple pattern and save as .bmp and .jpg file
#
# --- import modules we need
from PIL import Image, ImageDraw
import numpy as np
import scipy.misc
#
width = 1720
height = 670
channels = 3
#
img = np.zeros((height,
width, channels), dtype=np.uint8)
xx, yy = np.mgrid[:height, :width]
print "Generating Image with shape y: ", img.shape[0], " x: ", img.shape[1], " ... "
#
for y in range(img.shape[0]):
for x in range(img.shape[1]):
img[y] [x] [0] = int(y)
img[y] [x] [1] = int(x)
img[y] [x] [2] = int(y / 2.7)
# --- now, we have made the
image, lets save it
scipy.misc.imsave("testimg.jpg", img)
#
# --- Lets try to save it as a bmp (bitmap) file
scipy.misc.imsave("testimg.bmp", img)
#)
# --- This saves it in a .bmp format that Windows can read!
imgwtf = Image.open("testimg.bmp")
imgwtf.save("testimg.bmp")
#
# --- Read it back and display it on screen in Xwindows
image = Image.open("testimg.bmp")
image.show()
# --- Done.
--------------------------------------------------------------------------------------
I saved it as "mlsquare.py". You can run it using:
execfile("mlsquare.py")
from
within Python, or you can just enter, at command level...
python mlsquare.py
(If everything is ok, and you are in Xwindows, you will see
a bunch of
(pretty coloured squares, where each square has a different colour gradation.)
(It should also have saved two files: testimg.jpg and testimg.bmp There
is )
(example of using scipy and just the .save method for the image. To get an )
(image that Windows could read, I had to save it with scipy and then reload
)
(and save it. That hack worked, still not sure exactly the differences. )
(But what is interesting, is you can take this (and other) Python
programs, )
(and just copy them over to a Window machine, and they will run. And they )
(can do interesting and useful things. Python is kind
of amazingly good. )
Getting Jupyter and IPython to Run with Notebooks
=================================================
This is involved. You need to get and "git clone" the stuff
for "libzmq" and for
the library "pyzmq".
First, you build "libzmq". You only need to do this, if your Linux distro did not
have "libzmq". Mine did not.
To Do this, I did this (as root):
Build libzmq
============
mkdir /home/libzmq
cd /home/libzmq
- you need to get the source tree here. I used "git clone" and the
git clone url: "https://github.com/zeromq/libzmq"
- (actually, I lied: I used git clone first time on the L2-ACERT2
machine, and for here, I just copied the ../libzmq structure from
L2-ACERT2 to L2-ACERTM using "scp -r ..." )
- also, because I copied a whole subdir where I had run a cmake already
I had to delete the subdir "cmake-make" where I did the build...
rm -r -f cmake-make
- then, you have to make a fresh sub-dir and run "cmake" ...
mkdir cmake-make
cd cmake-make
cmake .. -G "Unix Makefiles"
- this will set things up for "make"
to run
make
make install
- this should build your libzmq and put it in the right place to be found
then...
Build pyzmq
===========
mkdir /home/pyzmq
cd /home/pyzmq
- you need to get the source tree here.
I used "git clone" and the
git clone url: "https://github.com/zeromq/pyzmq"
- (actually, I lied: I used git clone first time on the L2-ACERT2
machine, and for here, I just copied the ../libzmq structure from
L2-ACERT2 to L2-ACERTM using "scp -r ..." )
- also, because I copied a whole subdir where I had run a cmake already
I had to delete the subdir "cmake-make" where I did the build...
rm -r -f cmake-make
- then, you have to make a fresh sub-dir and run "cmake" ...
*** You might think to do
the same thing here... Don't. First,
*** Go to the URL "www.nelson-haha.com" and play it. For me...
*** The steps below are not valid. Once you get the libzmq, you can
just run pip to build pyzmq.
--- skip
this ---
mkdir cmake-make
cd cmake-make
cmake .. -G "Unix Makefiles"
- this will set things up for "make" to run
make
make install
- this should build your pyzmq and put it in the right place
to be found
--- end of skip ---
Do this instead:
I don't think you even
need to git get this source. Pip will likely do that...
Just run, from a terminal shell (might need to be root..)
pip install pyzmq
Installing Jupyter...
=====================
Basically, this is the "money-shot" that I had so much initial trouble with.
The idea is to Install "Jupyter" so that you can run "Notebooks" and draw
graphics inside of a a web-browser window.
This
is a brilliant idea, because if you have your browser working, then all
the GUI stuff should work to. As I understand it, Jupyter runs a web-server on
the "localhost" machine (ie. your own machine), and makes it available on port
number: 8888
If everything is ok, you should
just be able to install "Jupyter" with pip, and
then start "Notebooks" to have Interactive Python (IPython).
So, try to
run this:
pip install jupyter
You should see pip collect a few things... and
hopefully then report success.
Try to run Jupyter + Notebooks for Interactive Python:
======================================================
Start an Xwindows
session in a non-root (ie. standard user) account...
startx &
(I am using Fedora and Gnome. This should
work for any Linux, right?...)
(start a terminal shell...)
term
Start your Firefox web-browser...
I use a shell command:
runfirefox
Which just contains the following... (on L2-ACERT2)
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
/home/firefox34/firefox/firefox
Curiously, despite both machines having same /etc/ld.so.conf which has
/usr/local/lib
in it, L2-ACERTM runs without setting the path. If you
set the library path, and have installed the newer glib and gtk2, you
should be able to get the Firefox-34 binary stuff to run.
(You can enter the "runfirefox" command with a trailing "&" so it starts
and comes back to term shell, so you can enter "Jupyter Notebook", or
just put the three lines in a shell script. The idea is that as soon
as Jupyter starts, it will flip to the "localhost:8888/tree" url, and
show you your directory files... )
jupyter notebook
Ah, well --- this is why the log..
Works fine on L2-ACERT2, but first time trying this on
L2-ACERTM, and it
fails badly. I get an error saying:
Error: There was an error launching
the default action command
associated with this location.
This is a Jupyter error, and is printed in the term window.
I shutdown the Jupyter web server with a Control-C, and a "y" to confirm.
So far, no joy.
Fixing Jupyter Notebook to Open the Browser Window Immediately in Trusted Mode
==============================================================================
This looked to be some kind of security bullsh*t, or something weird
It did not present itself in the case of the first configuration. Why?
I realize security is critical, but this sort of stuff is just a fail.
I logged out, and restarted, and connected manually to the server from the
browser, by entering:
localhost:8888
In the Firefox URL bar. And I get a screen saying "Token
Authetication is
Enabled", which means maybe I got a *different* version of Jupyter today than 5
days ago? Arrrgh. (I checked. I did
not.) I will have to get the source, so I can
have stability... It looks like from the Nov. 30 or Dec. 2 version of Jupyter
to the version today (December 4, 2017), some sort of password authentication
has been enabled. Or, copying the git-got source tree from L2-ACERT2 to L2-ACERTM
created a second copy of some authentication bullshit, which then meant my
web-server autogenerated token was not accepted?
I noticed that there is a one hundred character random token string that seem
to be being passed from the Jupyter web-server, according to what is on the
command line.
This is maybe a design flaw? I now understand that using a web-server to serve up
information to a local session
is maybe not a great idea. It is great if your
goal is to sell "cloud services" to lazy people in commercial environments,
but if
your goal is research, running a local webserver is maybe an annoying
overhead. Now I have to figure out WTF I need to do to fix this.
<Later...>
Here is what I had to do for now...
From the Xterm window (the terminal command window Gnome desktop)
1) Highlight the "token" string that is shown when the webserver starts.
Do this with the mouse or trackpad on your laptop.
2) Once you have the text of the token highlighted, press left and right
mouse keys (or both left and right trackpad keys). This is same as
pressing the middle key on a three-button mouse. This copies the string.
3) Paste the token string into the password box by pressing the
right mouse key, then select "paste" option.
But this is a pain.
On my first laptop, when I start "jupyter notebook"
from a command shell, it
just opens the web-page directly, and the pages indicated: "Trusted"
On the version of Jupyter
built on the second laptop, this password/token copy
paste kudge is needed, and I get a "Not Trusted" on the notebook pages.
There seem to be some "notebook extensions" needed.
For now, I don't know what these are...
<Later still
- like, almost midnight!... >
****** ---> FIXED IT!
SO simple. In Linux, and Gnome, just start your Firefox Browser, and select
Preferences, and MAKE FIREFOX YOUR DEFAULT BROWSER.
==================================
Now, I also installed pygtk 2.11.0, but this did not seem to affect anything.
Note: to get
pygtk to install correctly, I had to get glib-2.0 to compile and
I moved the glib-2.0.pc file over to the /usr/lib/pkgconfig directory, so that
pkg-config now knows correctly the version of glib (2.26.1) I have built.
- you copy the glib-2.0.pc file from /home/glib to /usr/lib/pkgconfig
- pygtk configure was failing, because pkg-config was reporting
the 2.16.3 glib-2.0 package, but finding 2.26.1
glib-2.0 version.
- you check pkg-config package version with:
pkg-config --modversion glib-2.0
(returns 2.26.1)
- moved the /home/glib/glib-2.0.pc pkg-config config file into
the /usr/lib/pkgconfig directory. This let
pkg-config find the
right package details.
- then the /home/pygtk/pygtk-2.11.0 ./configure will run to completion,
and you can then run "make" and "make install". But this made no
difference to the startup problem. Setting Firefox to be the default
web-browser was what did it.
The key is to have a default web-browser defined in Gnome/X-windows desktop,
so that when you start a terminal shell, and enter
:
jupyter notebook
You get *immediately* taken into a web-browser page (the long "token"
is passed
automatically to the browser), and you are looking at a directory tree listing
of whatever directory you were in when you entered
the "jupyter notebook" command.
Fix for L2-ACERT2... (and L2-ACERTM...)
The stuff about re. pkg-config was useful, actually. I recompile glib-2.0, and
moved the glib-2.0.pc file to /usr/lib/pkgconfig.
This makes Firefox find the right damn libraries when it starts.
And this means, if Firefox is set as the "Default Browser", then just entering
jupyter notebook
on command line, will start the browser, and open the current directory tree as
a "localhost:8888/tree" in "trusted" mode. I don't have to set the browser's
LD_LIBRARY_PATH once I have the correct glib-2.0 libraries being found by the
Firefox binaries at startup time for Firefox. And now that Firefox is being
used as the "gui" window for Interactive Python and Notebooks, having Firefox
start correctly is critical.
Done - Jupyter/IPython works.
====
I now have the same Interactive Python environment running on both machines.
Confirmed by running histogram plot of 100 million pseudo-random numbers.
Will try to get Interactive Python running on the CentOS 6.6 box also.
Oh Sh(t!
========
Still cannot install or work with scikit-image. <big, long, sigh>...
Lets be clear: Free software is perhaps the most expensive option.
But Linux is the only thing I can use now, without
being
frustrated into a white-hot anger by modern vendor restrictions.
I just expect the computer to be a computer, you see.
With Linux, you can *always* get it done. YOu might have to
re-compile the kernel, or re-write something broken, but you
can *always* get it to run, since you've access to the source.
On Mars, we will all run Linux, since the vendor support
team
will be 40 million miles away best case, or 250 million worst
case.
More Stuff to Install:
======================
Ok, I have installed, using pip: scipy, numpy, matplotlib, and some other stuff.
But now, today (Dec. 6th, 2017) we are installing more (on CentOS first, just as a
test, since the CentOS box is running..)
<become
root... or the "pip install xxxx" command with generate a spew
of incomprehesible
track-back errors...>
pip install networkx
pip install six (this was already
done. "Pillow" was also installed.)
pip install PyWavelets
pip install
dask[array] (this is in numpy, but it installed the 'toolz' thing)
pip install scikit-image (this downloads and install a big .tar.gz file, 26.1 mb)
The objective here is again, to draw a damn picture and do something with it.
In this experimental case, I will just try to load a picture and make it grayscale,
from being a colour image.
There is some stuff to do this in scikit-image, but the above libraries were requisites
for scikit-image. If you have already installed
all the above libraries, you will see
the messages: "Requirement already satisfied ..." for seven of the scikit-image pre
requisites. You should also see:
Running setup.py bdist_wheel for scikit-image ... (tiny ascii throbber..)
(The tiny ascii throbber is a sequence of
characters: \ | - / -, that get displayed
at end of the above line, to indicate the setup process (ie. compile and install)
is running. This compile+install
activity can take several minutes. On my old
32-bit CentOS box, it means I go heat up my coffee in the microwave...)
Once the "scikit-image" completes
the install, you should see the message:
Successfully installed scikit-image-0.13.1
and that means you should
be able to start Python and import the "skimage" stuff.
I had recently created a colour test image of a bunch of squares, where each
square has a diagonal
gradation of colour. This code reads the testimg.png file,
and creates and displays a gray-scale version of the colour test file.
Try it:
# python
... (blah blah blah)
>>> from skimage.io import imread
>>> from skimage.transform import resize
>>> from matplotlib import pyplot as plt
>>> import matplotlib.cm as cm
>>> example_file = ("testimg.png")
>>>
image = imread(example_file, as_grey=True)
>>> plt.imshow(image, cmap=cm.gray)
>>> plt.show()
(right after you press return on the "plt.show()" command, you should see
the testimg.png file rendered as a gray-scale version, in a window, where
you can manipulate the image, zoom it, and otherwise mess about with it
a little.)
Oh, here is the code I used to generate the colour image of the many squares
with different colour gradations in each one.
-----------------------------------------------------------
Python program to create testimg.jpg
and testimg.bmp files.
-----------------------------------------------------------
#-------------------------------------------------------------------------------------
# --- Make and Show immediately a simple pattern and save as .bmp and .jpg file
#
# ---
import modules we need
from PIL import Image, ImageDraw
import numpy as np
import
scipy.misc
#
width = 1720
height = 670
channels = 3
#
img = np.zeros((height, width, channels), dtype=np.uint8)
xx, yy = np.mgrid[:height, :width]
print "Generating Image with shape y: ", img.shape[0], " x: ", img.shape[1], " ... "
#
for y in range(img.shape[0]):
for x in range(img.shape[1]):
img[y] [x] [0] = int(y)
img[y] [x] [1] = int(x)
img[y]
[x] [2] = int(y / 2.7)
# --- now, we have made the image, lets save it
scipy.misc.imsave("testimg.jpg", img)
#
# --- Lets try to save it as a bmp (bitmap) file
scipy.misc.imsave("testimg.bmp", img)
#)
# --- This saves it in a .bmp format that Windows can read!
imgwtf = Image.open("testimg.bmp")
imgwtf.save("testimg.bmp")
#
# --- Read it back and display it on screen in Xwindows
image = Image.open("testimg.bmp")
image.show()
# --- Done.
#--------------------------------------------------------------------------------------
If you want to run the above program as a test, just copy everything between
the dashed lines into a file, and call it "testpic.py". Then, just run
Python with that file. This should work on Windows, Macbook, and of course,
Linux, if you have setup
Python as indicated in my notes here. Remember, you
need to get Python 2.7.14, then you need pip, and you may have to build and
install a bunch of stuff from source on Linux to get
everything to work.
I just ran in on an old Windows-XP copy, with Python 2.7.12, and it ran fine.
c:\root\tfhack>python
... (blah, blah, blah)
>>> execfile("testpic.py")
(and you should see the image as a temporary .bmp file )
Since I am writing this on an old WinXP box, I figured I would try to load
scikit-image on the Windows box.
(run all the requisite "pip install xxxx" stuff above)
pip install scikit-image
I got everything,
except I scikit-image. The install for scikit-image
failed tests for "skimage._shared.geometry" extension, because I do not have
Microsoft Visual C++ 9.0 compiler. Now, there
is a *very helpful* error
message provided, saying I can get it from: "http://aka.ms/vcpython27".
(so, hell, lets have a look...)
Ok, usual
bs indicating hoo-haa, attempt to sell me a Surface Pro, and
stuff saying only for Windows xx. and later, etc... but if you drill down
and look around, you can find this: for "x86" boxes
(ie. 32-bit), and a
min. requirement set saying: Win 2000 SP4, Windows 7, Vista, Windows XP, ...
Ok, lets try this... (The URL here is:
https://www.microsoft.com/en-us/download/details.aspx?id=5582
And what this offers is:
Microsoft Visual C++ 2008 SP1 Redistributable Package
(x86)
Lets give it a whirl...
I dropped the file into: MS_Cplusplus_2008_sp1_vcredist_x86.exe
Take a copy of it (in case
it self-destructs, like the Adobe downloads do..)
From a Windows "Cmd" shell:
cd downloaded_software
copy MS_Cplusplus_2008_sp1_vcredist_x86.exe MSCpp.exe
MSCpp
And now, we re-try the:
pip install scikit-image
from the tfhack subdirectory.
Still no joy. The Microsoft install for the C++ redist. package did not say
to restart
Windows, but I should try that. Means I have to crash Firefox,
exit, and reboot. Oh, lets try it...
Ah, I also noticed there is a 2010 version of this file. I got
it, and
installed it too. It is the 2010 Microsoft C Runtime libraries. The URL is:
https://www.microsoft.com/en-us/download/confirmation.aspx?id=5555
I am guessing this stuff will be needed too...
Same fail... on building: 'skimage._shared.geometry' extension...
AH, I messed up: I got the runtime stuff, but did not get the compiler.
The compiler is here:
https://www.microsoft.com/en-us/download/confirmation.aspx?id=44266
It is big - 300 to 400 mb, completely unsupported, blah blah. It comes
down as a .msi file. Try to run it...
It says its: Microsoft C++ Compiler Package for Python 2.7.
Wow. I will be surprised if this works...
Another try at "pip install scikit-image" ...
Yup. Complete kack. But it got further. It appears to have compiled
the geometry.c, but failed at C:\WATCOM\H\io.h (WHAT? You guys are
using the Watcom C compiler, eh? Good on ya! I used to drink *beer*
with the Watcom guys, back in the old days... )
So, why won't this compile?? I'm getting: C4068
unknown pragma.
And then about 100
"Error C2485: '__watcall' " unrecognized extended attribute
which looks like its
coming from: stat.h(218)
and then stat.h(219), 221, 222, 223, 234, 225 and so on...
Huh? Once the error count ticked over
100, compilation commited
suicide. Hmmm. Wait a minute! Why is Watcom compiler being mixed
with Microsoft C++ version 9? That is nuts.
I know what happened,
I have C:\WATCOM installed, and the compile process found something
in the path, and used it that it should not probably have done!
I renamed C:\WATCOM to C:\This_is_WATCOM_stuff, so I did not have to
mess with my path defintions. WATCOM is not used here, I am sure...
Gotta be that. This time "pip
install scikit-image" gave me a
whole bunch of successful compiles, but failed on tifffile.c(75), as
it is unable to find stdint.h, saying:
'stdint.h': No such file or directory.
Turns out some dude named "Kyle" forgot to put a stdint.h thing in
somewhere. This is a "numpy" core routine that is failing to
compile. Yeah, that's gotta be it. I have "Kyles_stdint.h" which
I found via a google-search. It has to be put somewhere where the
compiler can find it, and the Nov.
2017 note (which is specific for
WIndows version 10,000 or whatever the hell version they are using now),
is not going to work for me. Hmmm. How about the VC "include"
subdir?
I put Kyle's "stdint.h" into the "include" subdir in the VC directory
for the special Visual C++ 9.0 for Python dir. The full dir-name on
a Window-XP/sp3 box is:
C:/Program Files/Common Files/Microsoft/Visual C++ for Python/9.0/VC/include/
And Whoo-hoo! It all
ran. I got a clean compile and install of the
scikit-image stuff (which seems to have rebuilt "numpy"). The command
pip install scikit-image
reports:
Successfully installed scikit-image-0.13.1
Ok, lets see if the little program to turn image colour to gray,
works...
On the Windows box, running Python 2.7.12 ... (after pip installing stuff)
python
... (blah and blah)
>>> from skimage.io import imread (first line worked... a good sign)
>>> from
skimage.transform import resize
>>> from matplotlib import pyplot as plt
>>> import matplotlib.cm as cm
>>> example_file = ("testimg.bmp")
>>> image = imread(example_file, as_grey=True)
>>> plt.imshow(image, cmap=cm.gray)
>>> plt.show()
( you should see a window with a gray-scale image)
And, you should see a window
appear with a gray-scale version of the file
called "testimg.bmp" (On Linux, I used a .png file, and on Windows, a .bmp
file, but they should work the same way.) So, we have
a nice program suite
for image manipulation that works isometrically on Linux and Windows machines.
I have to say, Python 2.7, and it's suite of libraries, represents a *very*
impressive step forward in computer technology. The entire paradigm leverages
the sensible stuff that came before, without getting silly. It is possible to
get some real work
done, without first getting drowned in nonsense.
And of course, given that I had the compiler and the image stuff working,
I had to try:
pip install jupyter
and it worked. Everything installed, and the compiles all ran to completion
on the Windows box (an old Windows-XP/Sp3 box, with many patches and upgrades).
Confirmed that the .ipynb (notebook files) from Linux can be simply scp'ed
from Linux box to Windows box and the histogram and line-graph examples run.
I was able to run, from a "Cmd" shell:
jupyter notebook
and the Firefox browser opened a tab, with a url of "http://localhost:8888/tree",
showing the directory structure of the directory I launched from. It appeared
to work exactly the same as the Linux version.
Bravo, guys. This is cool.
- M. Langdon, GEMESYS Ltd. - Dec. 6, 2017
======================= =============
PS: Thank-you, Kyle, for putting your note on StackExchange. And thank-you Matt,
and also Alexander Chemeris who appears
to be the author of the "stdint.h" thing
that was needed. And of course, thank-you to everyone who made this stuff, at
every level. It's all awesome. If you are building scikit-image with pip
install, just grab the "stdint.h"
and put it in the "include" lib for the VC
(Visual C++ 9.0 compiler for Python 2.7) that Microsoft lets you download.
I had no plans to get all this working for Windows, but it is actually pretty
cool to have it working seemlessly on both Windows
and Linux. Thanks a ton also
to Microsoft for releasing the Visual C++ 9.0 for use with Python 2.7.
And of course, thank-you to the author of scikit-image.
The missing "stdind.h" for scikit-image compile on Windows is at:
https://raw.githubusercontent.com/mattn/gntp-send/master/include/msinttypes/stdint.h
Tck/Tk (matplotlib) stuff to show images on Linux laptops does not work!
========================================================================
You think you are done, but you never are...
Had to build, make and "make install" the Tcl an Tk sources for Tck/Tk 8.5
I start Python on the Linux laptops,
and find I cannot do what works on the
CentOS and Windows versions of Python 2.7.14, namely draw an image to screen,
without the "jupyter notebook" thing. If I start Python, and do this:
>>> import matplotlib
>>> matplotlib.get_backend()
I get:
u'agg'
which means I have *no* "tkinter" - the Tcl/Tk interface, which is supposed
to be built-in. Maybe because I
have Tcl/Tk version 7.3 installed, so as to
let Xerion work (it needs Tcl/Tk 7.3)
I rebuild Tck/Tk 8.5, and fiddle about with the /usr/local/bin and /usr/local/lib
stuff so that Tck/Tk 8.5 (and wish 8.5) works.
And I try to rebuilt Python27 in the /home/Python27 dir, as root...
python setup.py build
python setup.py install
And I start Python, and see what I have ...
>>>
import matplotlib
>>> matplotlib.get_backend()
u'agg'
Bugger.
But, I did the build, and it looked ok, and I have Tcl/Tk installed, and actually
have written real apps that use Tcl/Tk - serious stuff, editors, encrypters, etc.
So, I have Tcl/Tk working, I know. But I had a previous version installed, to run
the Xerion stuff. I rename the /usr/local/bin stuff so tcl and tclsh points to the
Tck/Tk 8.5 version (not the
7.3 version used by Xerion). And then, I recompile
and reinstall, Tck/Tk, from my source directory, on /home/TckTk. Once for Tcl 8.5
and also for Tk 8.5. This probably updates pkg-config,
and puts the libs with the
right links in /usr/local/lib. I run "ldconfig" from root.
Now, lets try this: (I am trying to draw my little gray-scale image on the
Gnome/Xwindows screen)
(start Python 2.7.14)
>>> import _tkinter
(this works without complaining)
>>> import matplotlib (this complains about _posixsubprocess module not
being used, but subprocess32 is installed. Looks to
be a bug, but does not seem to affect anything)
>>> matplotlib.rcParams["backend"] = "TkAgg" (set matplotlib backend to TkAgg)
(and confirm it got set...)
>>> matplotlib.get_backend()
u'TkAgg'
(ok, we set the backend parameter)
>>> execfile("drawgray.py") (This
is just the 8-line test file from above, which
draws the colour test image "testimg.bmp" in gray-
scale shading. Modified program is below...)
!! And it WORKS! Great! The laptop Linux (based on a Fedora-9 kernel, originally,
reports from the "uname -a" as: Linux 2.6.24-14.fc9.i686.
This is cool. I have the two Linux laptops working the same as the CentOS box, and the
Windows P/C. I suppose the MacBook is next...
Here is a copy of the little test program, confirmed it works on all three platforms now:
Drawgray_linux.py
--------------------------------
#-----------------------------------------------------------------------------
#--- draw grey image from colour image
#
#--- this lets us use the cool Tcl/Tk stuff for images
import _tkinter
#--- matplotlib.get_backend() => report u'agg' which is no graphics
#
unless we use jupyter notebook stuff
#--- if the _tkinter import worked, we can draw pictures with Tcl/Tk
import matplotlib
matplotlib.rcParams["backend"] = 'TkAgg'
#--- this lets us use the cool Tcl/Tk stuff for images
from
skimage.io import imread
from skimage.transform import resize
import matplotlib.cm as cm
from
matplotlib import pyplot as plt
#
#--- get colour example file
example_file
= ("testimg.png")
image = imread(example_file, as_grey=True)
plt.imshow(image, cmap=cm.gray)
#--- show image on screen
plt.show()
#-----------------------------------------------------------------------------
Bottom line: The re-compile and install of Tck/Tk, and the rebuild and re-install of the
Python 2.7.14, allowed the Tck/Tk stuff to be found and used on the old Linux laptops.
They may be 32-bit, but they can run 100 million simulations,
where random vars are
drawn from a normal-distribution, calcs are done, and a detailed stacked histogram is
generated, and all this gets done and displayed as an image, in about 15 seconds.
Hope this is useful for someone...
:D
- M.