Example Python Program to Create Fully-Connected Graph

#--- draw fully connected graph example using Tck/Tk
import numpy as np
import _tkinter
# --- turn off warnings
import warnings
warnings.filterwarnings('ignore')

import matplotlib
matplotlib.rcParams["backend"] = 'TkAgg'

import matplotlib.pyplot as plt
import networkx as nx

# --- ask user for size of graph to draw...
print " "
print "Create Fully-Connected Graph."
print " "
gsize = int(input("Enter the number of nodes to draw: "))
gvec = []
for i in range(gsize):
    gvec.append(i+1)

g = nx.Graph()
# --- g = nx.cycle_graph(1)

g.add_nodes_from(gvec)
# A = nx.adjacency_matrix(G)

# --- gsize = 7
edgectr = 0
for a in range ( 1, (gsize+1)):
    for b in range ((a+1), (gsize+1)):
        g.add_edge(a, b)
        edgectr = edgectr+1
# ---     print "A is: ",a, "   B is: ",b


# --- g.add_edge(1, 7)

nx.draw_networkx(g)
print "Drawing fully-connected graph, with: ",gsize," verticies"
print "Total number of links created was:  ",edgectr
print "... (close graph-window to exit...) "

plt.title("Fully Connected Graph - Nodes: "+str(gsize))
plt.show()

# --- done.

 

Example of Fully-Connected Graph, created with Python, and _tkinter, matplotlib, networkx and numpy libraries.

This is a .jpg image which simulates Moire patterns, and was created on a MacBook, using a Python program and the scipy and numpy (the scientific and numeric Python libraries) and PIL (the Python Image Library, but now actually called Pillow). This image was rendered using the Preview utility of the MacOs. The Python program that generated it is shown below.

Example Python Program to Create Simulated Moire Pattern

#
# --- Make and Show immediately a Moire pattern as a .jpg file
#
# --- For now, you start Python and import this file with: import MakeShowMoire
# --- or use: execfile ("MakeShowMoire.py")
#
# --- import modules we need
from PIL import Image, ImageDraw
import numpy as np
import scipy.misc
#
# --- Ok, lets go..
width = 1020
height = 670
channels = 3
#
img = np.zeros((height, width, channels), dtype=np.uint8)
xx, yy = np.mgrid[:height, :width]
circle = .2 * ( (xx - 1000) ** 2 + (yy + 1000) ** float(1.9795) )
print "Generating Image with shape y: ", img.shape[0], "  x: ", img.shape[1], " ... "
#print "xx is: ", xx
#print "yy is: ", yy
#
for y in range(img.shape[0]):
    for x in range(img.shape[1]):
        r, g, b = circle[y][x], circle[y][x], circle[y][x]
        img[y] [x] [0] = r
        img[y] [x] [1] = g
        img[y] [x] [2] = int(y / 2.7)

#
# --- now, we have made the image, lets save it
scipy.misc.imsave("testimg.jpg", img)
#
# --- Since we have saved it, we should be able to display it
image = Image.open("testimg.jpg")
image.show()

Xerion Display of 30-40-1 neural network, showing output nodes. Running on Fedora Linux, Gnome X11 desktop. Code to define network and load training cases is provided below.

Code Examples: (Code is experimental, for use-at-own-risk, of course. )

Code examples:

Tck/Tk "CompareMarketNet.tcl" Evaluation Program

This simple program just processes the evaluation cases (same format and name of the original training cases used to train the network), and calculates a simple "co-efficient of accuracy", which is just the percentage of network outputs that successfully match the target.  Since the network trains to, and also outputs signed boolean values, any value in the range of the absolute value of .8 or less is counted as a zero, for evaluation purposes.  If the network generates a -.723 or a +.6824, that gets evaluated as a zero.  If the network generates a .9124, that gets evaluated as a +1.0 boolean, and so on.

#
# --- run a compare between MarketNet "tcasetab.txt" Training set and trained network output
# --- also, calculate a "coefficient of accuracy", basically the % right
#
#
proc compareMarketNet {} {
   puts "MarketNet Network:  Target versus Output from network... "
   set rightctr 0
   set totctr 0
#
   loop i 0 [MNTraining size]  {
     uts_presentExample MarketNet MNTraining $i
     uts_activateNet MarketNet
     set cdate  [MNTraining  tag $i]
     set output [lindex [MarketNet.Output.0 configure -output] 2]
     set target [lindex [MarketNet.Output.0 configure -target] 2]
     puts "$cdate ::   MarketNet target: $target,  MarketNet output: $output"
     set rndoutput round($output)
     set rndtarget round($target)
     set absvalout abs($output)
     if {[expr [expr $rndoutput==$rndtarget] || [expr [expr $target==0.0] && [expr $absvalout<=0.80]]] }  {
    set rightctr [expr $rightctr+1]
        }
     if {$target != -999} {
        set totctr [expr $totctr+1]
        }
  }
   puts "--- < Done > --- "
   puts "Number Right: $rightctr"
   puts "Total Cases : $totctr"
   puts "Coeff.Accur : [expr 1.0 * $rightctr/$totctr]"
}

Building Neural Networks with Xerion:

The example here is for defining a Neural Network using Xerion,  loading the training cases, and defining a tcl/tk program to plot the actual vs. predicted.  Network is a 30-40-1 topology, the current MarketNet example used to predict whether the 5-day ahead price will be up up more than 1% (output = +1), no significant change (output = zero), or down more than 1% (output = -1).  You can download Xerion and tcl/tk (and the various sub-components needed, such as tclX, itcl and tkmxm) from url:  http://ftp.cs.toronto.edu/dist/xerion/

To compile Xerion, you may have to make a number of tweaks to get it to run on your local flavour of Linux.  If you are using modern C compilers (I used an old gcc 4.3.0, circa 2008), you will need to include a varargs.h function, to allow variable number of args be passed.  I used an old Xenix header, from a little-known left-coast software firm called Microsoft...).  Here is a url to a copy of it, from the Arecibo Observatory website http://www.naic.edu/~phil/hardware/vertex/sharemegsvertex/ocu/pcr/mscv51/include/VARARGS.H    This code snippet is dated 1985-1988, and has a Microsoft copyright stmt in it.  Without the copyright notice, it is nine lines of code (8 lines, if you don't count the whitespace.)   You will need to include a varargs.h into order to get Xerion to compile with a modern gcc. 

Defining the Network, Loading Training Cases:  (file: mktnet3040.in). 

This file can be "sourced" into Xerion at the Xerion command line (at "bp-> " prompt), or from the Xerion gui screen, under the "File" option.  It is basically bash + tcl/tk code.

             

   ###################################################################################
   # A single hidden layer (40 units) net for Cross-sectional Time Segment MarketNet #
   #                                                                                 #
   # Training Cases built by MAKECASE, an APL workspace, with function: MAKECASE     #
   # Database of Market Prices in: /home/cas/MARKET/DATA/timesr1.sf                  #
   # File "tcasetab.txt" created in ../MARKET/DATA, than copied to                   #
   #       directory /home/cas/marketAI                                              #
   #                                                                                 #
   #                                                                                 #
   # '$Id:  2017/05/17  mlangdon Exp $'                                              #
   ###################################################################################

   set net    MarketNet ;

   catch {
     $net delete ;
   }

   # --- Create a network with 30 inputs, 40 hidden units, 1 output unit
   # --- this network will use TANH activation functions
   #
   uts_simpleNet $net 30 40 1 ;

   bp_groupType MarketNet.Hidden {HIDDEN TANH}
   bp_groupType MarketNet.Output {OUTPUT TANH}

   random seed 3 ;
   uts_randomizeNet MarketNet ;

   # Add a new example set for training
   set exampleSet "MNTraining" ;
   catch {
     $exampleSet delete ;
   }

   uts_exampleSet   $exampleSet ;
   uts_loadExamples $exampleSet  /home/cas/marketAI/tcasetab.txt

   #
   # Procedure for plotting network output and target
   #

   proc plotValues { net exampleSet} {

     set t [open "target.dat" w] ;
     set o [open "output.dat" w] ;
     set cr \r;
     set blk "  ";
     set endnum [$exampleSet size];
     loop idx 0 $endnum {
       uts_presentExample $net $exampleSet $idx ;
       uts_activateNet $net ;

      regexp {([0-9]+)} [$exampleSet tag current] ddate ;
   #   if { $idx >= $strpnt } {
       puts $o "$idx $blk [lindex [$net.Output.0 configure -output] 2] $blk $ddate $cr";
       puts $t "$idx $blk [lindex [$net.Output.0 configure -target] 2] $blk $ddate $cr";
   #   }
     }
     close $t ;
     close $o ;

     set command {
       set terminal X11 ;
       set grid ;
       set title "** Scaled ** MarketNet: CM Close in Boolean Jumpdeltas - Daily"
       set xlabel "Date Index Position" ;
       set ylabel "CM JumpDeltas " ;
       set style data lines
   #--- old format:    set data style lines
   #---  set yrange [.50:.800]
       plot "target.dat" title "Actual CM",  \
          "output.dat" title "Predicted CM"
       pause -1 "Hit Return to quit, and run Postscipt prt to outfile" ;
       set term postscript ;
       set output "outfile" ;
       plot "target.dat" title "Actual CM",  \
          "output.dat" title "Predicted CM"
       pause -1 "Hit return to quit" ;
       }
     exec echo $command > plot.gnu
     exec gnuplot plot.gnu >@ stdout 2>@ stdout ;
   }

 < This file is "compareMarketNet.tcl", and is used to compare training target and actual network output>

#
# --- run a compare between MarketNet "tcasetab.txt" Training set and trained network output
proc compareMarketNet {} {
   puts "MarketNet Network:  Target versus Output from network... "
   loop i 0 [MNTraining size]  {
     uts_presentExample MarketNet MNTraining $i
     uts_activateNet MarketNet
     set cdate  [MNTraining  tag $i]
     set output [lindex [MarketNet.Output.0 configure -output] 2]
     set target [lindex [MarketNet.Output.0 configure -target] 2]
     puts "$cdate ::   MarketNet target: $target,  MarketNet output: $output"
   }
   puts "--- < Done > --- "
}

 

Python & TensorFlow & PIL (Python Image Library) to display the exploding stars example.

Second piece of the code is the "raindrops" laplace PDE (partial differential equations) simulation, using Python 2.7 and TensorFlow, Numpy, Scipy and PIL (actually Pillow), on a MacBook.  (See screen image from Macbook below.)  The code is from Google TensorFlow tutorials.   The original code uses Python Notebooks, Jupyter, and some other Python extentions, so the simulation can be seen to evolve in real-time - which is of course really cool.  But it is also nice to get something actually running, just to checkpoint one's progress.   This code below will produce the five images shown in the image to the right, and will confirm that you have the baseline configuration needed to get TensorFlow running on your Mac OSX.  I used Python 2.7.11, because I had it there already.  The MacBook OSX version is Yosemite (OSX 10.10.5).  TensorFlow needs Python 2.7.x, at a minimum. (The MacBook came with Python 2.6 installed, which needed to be upgraded.)

 

    # --- the Tensorflow LaPlace Image example (Uses PIL, and scipy.misc)
    #
    # --- Modified: Mar 7, 2017 - by MCL, to just use image file display
    # ---                                       instead of Python Notebooks, IPython, etc.,
    # ---                                       with negative damping and darker image backgrd.
    # ---                                       (Instead of ripples in a pond, we have
    # ---                                       exploding stars ... )
    # --- Produces Initial image, 3 intermediated images, and the final image
    #     as .jpg files. Requires only: tensorflow, numpy, scipy and Pillow
    #     and Python 2.7.10.
    # --- This example taken from Tensorflow Site:
    # ---                                     https://www.tensorflow.org/tutorials/pdes
    # --- and provides a nifty example of manipulating n-dimensional tensors.
    # ---
    # --- For Python newbies (me!):   1) invoke Python in terminal shell
    # ---                             2) >>> execfile("LaplaceMCL.py")
    # --- focus on understanding exactly how Tensorflow is reshaping tensors
    # ------------------------------------------------------------------------------------------
    # --- Import libraries for simulation
    import tensorflow as tf
    import numpy as np
    import scipy.misc

    # --- Import for visualization (works now - jpeg encoder recognized
    # import PIL.Image
    from PIL import Image, ImageDraw
    from io import BytesIO
    # xxx from IPython.display import clear_output, Image, display

    # --- function for displaying state of the pond's surface as an image
    def DisplayArray(a, fmt='jpeg', rng=[0,1]):
       """Display an array as a picture. """
       a = (a - rng[0])/float(rng[1] - rng[0])*37
       a = np.uint8(np.clip(a, 0, 255))
       f = BytesIO()
       # PIL.Image.fromarray(a).save(f,fmt)
       # clear_output(wait = True)
       # display(Image(data=f.getvalue()))
       # --- write the image
       scipy.misc.imsave("tensor.jpg", a)
       image = Image.open("tensor.jpg")
       image.show()

    # --- make interactive session for messing around - can use regular session also
    sess = tf.InteractiveSession()
    # sess = tf.Session()

    # --- computational functions go here...
    def make_kernel(a):
       """Transform a 2D array into a convolutional kernel """
       a = np.asarray(a)
       a = a.reshape(list(a.shape) + [1,1])
       return tf.constant(a, dtype=1)

    def simple_conv(x, k):
       """ A simplified 2D convolutional operation """
       x = tf.expand_dims(tf.expand_dims(x, 0), -1)
       y = tf.nn.depthwise_conv2d(x, k, [1, 1, 1, 1], padding='SAME')
       return y[0, :, :, 0]

    def laplace(x):
       """Compute the 2D laplacian of an array """
       laplace_k = make_kernel([[0.5, 1.0, 0.5],
                                           [1.0, -6., 1.0],
                                           [0.5, 1.0, 0.5]])
       return simple_conv(x, laplace_k)

    # --- Define the PDE - the pond is a perfect 500x500 square
    N = 500

    # --- now, we create some raindrops
    # --- Initial Conditions -- some rain drops hit the pond
    # --- set everything to zero
    u_init = np.zeros([N, N], dtype=np.float32)
    ut_init = np.zeros([N, N], dtype=np.float32)

    # Some raindrops hit pond at random points (or stars in a starfield)
    for n in range(40):
       a,b = np.random.randint(0, N, 2)
       u_init[a,b] = np.random.uniform()

    # --- Create and Display the jpeg image...
    DisplayArray(u_init, rng=[-0.1, 0.1])

    # Parameters
    # eps -- time resolution
    # damping -- wave damping
    eps = tf.placeholder(tf.float32, shape=())
    damping = tf.placeholder(tf.float32, shape=())

    # --- Create vaiables for simulation state
    U = tf.Variable(u_init)
    Ut = tf.Variable(u_init)

    # --- Discretized PDE update rules
    U_ = U + eps * Ut
    Ut_ = Ut + eps * (laplace(U) - damping * Ut)

    # --- Operation to update the state
    step = tf.group(
       U.assign(U_),
       Ut.assign(Ut_))

    # --- Run the simulation forward with a simple FOR loop.
    # --- Initialize sate to initial conditions
    tf.global_variables_initializer().run()

    # --- Run 1700 steps of PDE
    for i in range(1700):
       # Step simulation (damping was 0.04, I made it negative .14)
       step.run({eps: 0.03, damping: -0.14})
       if (i == 400) : DisplayArray(U.eval(), rng=[-0.1, 0.1])
       if (i == 1000) : DisplayArray(U.eval(), rng=[-0.1, 0.1])
       if (i == 1400) : DisplayArray(U.eval(), rng=[-0.1, 0.1])

    # --- Ok, we are done...
    DisplayArray(U.eval(), rng=[-0.1, 0.1])

    # --- show the jpeg. Not needed if using DisplayArray.
    # image = Image.open("tensor.jpg")
    # image.show()


 

Questions to gemesyscanada at gmail dot com, if curious.  

 

sAPL code: 

I've imaged some of the sAPL page-frames, to show terse, useful sAPL code (reads in a text file, processes it into 32-bit floats, plus some binary-ascii conversion examples.  One can read the Xerion network weights file into sAPL, and use tanh and dot-product multiply to activate (ie. "run") the Xerion-trained neural network.  See images for sAPL code below]I

The sAPL interpreter, as an .exe, is available on the GEMESYS Github account.  It is a full APL interpreter, it has no in-app adverts, and is free, and may be copied freely.  It is the original IP Sharp APL, running in an emulator for IBM 370 assembler.  It runs in a DOSbox, itself also an emulator, but this means sAPL can be run on Windows under cmd shell, on Linux using DOSemu, on an iPad using DOSpad, and it is available at no charge, (no tracking or in-app adverts)  from Google PlayStore (search for "GEMESYS" and select "sAPL" and download to any Android device.)

The examples here show how it can be used to "run" the network which has been trained using Xerion, by loading in the high-precision (17 digits real) ascii-text network weights file, and using those weights to process a boolean training case.  The sAPL interpreter, which is running on an "jailbroken" iPad,  (which is running  gDOSpad - the custom version of DOSbox for the iPad) is used to do this.  The example screenshots show the APL functions that do this.  In this way, the iPad can be used as a portable AI executor, to examine real-time boolean jump-delta vectors (driven by the market prices), and provide predictive information that is actionable.

 

TensorFlow+Python+Numpy+Scipy+PIL(actually PIL is Pillow) Laplace Simulation of Exploding Stars.

This is sAPL function to read a raw ascii text file of Xerion weights into sAPL as a vector of chars.

This procwt function takes as parameter the raw weights char. vector, and processes it into high-precision 32-bit reals for calculaton.

Utility functions to create pointer vector, raise negatives signs, and execute hyperbolic tangent activation transfer function.

Some examples of translate functions, overlaid on image of our homeworld, imaged from the International Space Station. The binary table bottom-right, is courtesy of xkcd site. I learned APL first, and I tend to use it to pseudo-code a problem. But once pseudo-coded, it can usually just be transcribed into a runable program. When machines were slow, performance was an issue. Now, performance (execution-speed) is not an issue, typically. It is the human-performance that is causing issues now (pretty much as it always has been, throughout all of history.)

The Actnet example to Activate the Xor2 Network.

Actnet processes Wtab1, Wtab2 and Btab. Wtabs are network weights, and Btab are bias values. This screencap shows the result, which matches exactly the Xor2 net activated using Xerion. Note the Xor2 net uses logistic transfer function (code provided previous image).

Proof-of-Concept Prototype: (or "The Money Shot"!) This is Actnet2, running against a full boolean jump-delta vector case (created by MAKECASE), and generating the same results as the Xerion-trained neural network. Wtab1 and Wtab2 are the network weights data. Btab are the node bias values. Function Actnet2 runs the Xerion-trained network, for the two boolean "Example Case" vectors, and generates and displays network output. This demonstrates the feasibility of the "Helper AI" concept, using existing technology. Market price-changes, beyond a certain level, can be subjected to a quick evaluation by a tablet device, and action can be taken, if appropriate. The AI can offer immediate, actionable advice, the accuracy of which can be subject to formal evaluation. That way, you can determine if it is actually providing a real edge.

Montage image comparison of two different networks, (left window is newest network, right window is original network). The two networks were trained on same dataset, and are here shown run against the small evaluation training cases (shown in middle screen). The two different neural networks are reporting different results, which suggests that training down to a low sum-squared error level does not necessarily make for predictability. Original network seems to offer a more accurate forecasting outcome. This shows example of why training to lowest sum-squared error is not necessarily as good as training to other objectives, such as maximized cross-entropy.