news

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 FOSS documentation emacs fedora foss freedom gnome haskell install laptop photo ruby travel verilog vhdl vlsi xmonad


[Published in Open Source For You (OSFY) magazine, June 2015 edition.]

In this final article in the Haskell series, we shall explore how to use it for web programming.

Scotty is a web framework written in Haskell, which is similar to Ruby’s Sinatra. You can install it on Ubuntu using the following commands:

$ sudo apt-get install cabal-install
$ cabal update
$ cabal install scotty

Let us write a simple `Hello, World!’ program using the Scotty framework:

-- hello-world.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    html "Hello, World!"

You can compile and start the server from the terminal using the following command:

$ runghc hello-world.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

The service will run on port 3000, and you can open localhost:3000 in a browser to see the `Hello, World!’ text. You can then stop the service by pressing Control-c in the terminal. You can also use Curl to make a query to the server. Install and test it on Ubuntu as shown below:

$ sudo apt-get install curl

$ curl localhost:3000
Hello, World!

You can identify the user client that made the HTTP request to the server by returning the “User-Agent” header value as illustrated in the following example:

-- request-header.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty

main :: IO ()
main = scotty 3000 $ do
  get "/agent" $ do
    agent <- header "User-Agent"
    maybe (raise "User-Agent header not found!") text agent

You can execute the above code in a terminal using the following command:

$ runghc request-header.hs  
Setting phasers to stun... (port 3000) (ctrl-c to quit)

If you open the URL localhost:3000/agent in the browser, it returns the following User-Agent information on Ubuntu 14.10 Mozilla/5.0 (X11; Linux x8664) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/41.0.2272.76 Chrome/41.0.2272.76 Safari/537.36. The Curl version is returned for the same URL request as shown below:

$ curl localhost:3000/agent -v

 * Hostname was NOT found in DNS cache
 *   Trying 127.0.0.1...
 * Connected to localhost (127.0.0.1) port 3000 (#0)
 > GET /agent HTTP/1.1
 > User-Agent: curl/7.37.1
 > Host: localhost:3000
 > Accept: */*
 > 
 < HTTP/1.1 200 OK
 < Transfer-Encoding: chunked
 < Date: Wed, 29 Apr 2015 07:46:21 GMT
 * Server Warp/3.0.12.1 is not blacklisted
 < Server: Warp/3.0.12.1
 < Content-Type: text/plain; charset=utf-8
 < 
 * Connection #0 to host localhost left intact

 curl/7.37.1

You can also return different content types (HTML, text, JSON) based on the request. For example:

-- content-type.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty as W
import Data.Monoid
import Data.Text
import Data.Aeson

main :: IO ()
main = scotty 3000 $ do
  get "/hello" $ do
    html $ mconcat ["<h1>", "Hello, World!", "</h1>"]

  get "/hello.txt" $ do
    text "Hello, World!"

  get "/hello.json" $ do
    W.json $ object ["text" .= ("Hello, World!" :: Text)]

You can start the above server in a terminal as follows:

$ runghc content-type.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

You can then open the three URLs listed above in a browser to see the different output. The respective outputs when used with Curl are shown below:

$ curl localhost:3000/hello
<h1>Hello, World!</h1>

$ curl localhost:3000/hello.txt
Hello, World!

$ curl localhost:3000/hello.json
{"text":"Hello, World!"}

You can also pass parameters in the URL when you make a request. The param function can be used to retrieve the parameters as indicated below:

-- params.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty
import Data.Monoid

main :: IO ()
main = scotty 3000 $ do
  get "/user" $ do
    name <- param "name"
    html $ mconcat ["<h1>Hello ", name, "</h1>"]

You can start the above server using the runghc command:

$ runghc params.hs 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

You can now try the URL requests with and without parameters. The observed outputs are shown below:

$ curl localhost:3000/user
<h1>500 Internal Server Error</h1>Param: name not found!

$ curl localhost:3000/user?name=Shakthi
<h1>Hello Shakthi</h1>

The Hspec testing framework can be used for integration testing the web application. Install the required dependencies as shown below:

$ cabal install happy hspec hspec-wai hspec-wai-json

The content type example has been updated to use Hspec, as illustrated below:

-- content-type-spec.hs

{-# LANGUAGE OverloadedStrings, QuasiQuotes #-}
module Main (main) where

import Data.Monoid
import Data.Text

import           Network.Wai (Application)
import qualified Web.Scotty as W
import           Data.Aeson (object, (.=))

import           Test.Hspec
import           Test.Hspec.Wai
import           Test.Hspec.Wai.JSON

main :: IO ()
main = hspec spec

app :: IO Application
app = W.scottyApp $ do
  W.get "/hello.txt" $ do
    W.text "Hello, World!"

  W.get "/hello" $ do
    W.html $ mconcat ["<h1>", "Hello, World!", "</h1>"]

  W.get "/hello.json" $ do
    W.json $ object ["text" .= ("Hello, World!" :: Text)]

spec :: Spec
spec = with app $ do
  describe "GET /" $ do
    it "responds with text" $ do
      get "/hello.txt" `shouldRespondWith` "Hello, World!"

    it "responds with HTML" $ do
      get "/hello" `shouldRespondWith` "<h1>Hello, World!</h1>"

    it "responds with JSON" $ do
      get "/hello.json" `shouldRespondWith` [json|{text: "Hello, World!"}|]

You can compile the above code as shown below:

$ ghc --make content-type-spec.hs     
Linking content-type-spec ...

The following output is observed when you run the above built test executable:

$ ./content-type-spec 

GET /
  responds with text
  responds with HTML
  responds with JSON

Finished in 0.0010 seconds
3 examples, 0 failures

Please refer to the hspec-wai webpage at https://github.com/hspec/hspec-wai for more information.

Template support is available through many Haskell packages. The use of the blaze-html package is demonstrated below. Install the package first using the following command:

$ cabal install blaze-html

Consider a simple web page with a header and three unordered lists. Using blaze-html, the template can be written in Haskell DSL as follows:

-- template.hs

{-# LANGUAGE OverloadedStrings #-}

import Web.Scotty as W
import Text.Blaze.Html5
import Text.Blaze.Html.Renderer.Text

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    W.html . renderHtml $ do
      h1 "Haskell list"
      ul $ do
        li "http://haskell.org"
        li "http://learnyouahaskell.com/"
        li "http://book.realworldhaskell.org/"

You can compile the above code using GHC:

$ ghc --make template.hs 
Linking template ...

You can then execute the built executable, which starts the server as shown below:

$ ./template 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

Opening a browser with URL localhost:3000 will render the expected HTML file. You can also verify the resultant HTML output using the Curl command as shown below:

$ curl localhost:3000
<h1>Haskell list</h1><ul><li>http://haskell.org</li><li>http://learnyouahaskell.com/</li><li>http://book.realworldhaskell.org/</li></ul>

It is good to separate the views from the actual application code. You can move the template content to a separate file as shown below:

-- Haskell.hs

{-# LANGUAGE OverloadedStrings #-}

module Haskell where

import Text.Blaze.Html5

render :: Html
render = do
  html $ do
    body $ do
      h1 "Haskell list"
      ul $ do
        li "http://haskell.org"
        li "http://learnyouahaskell.com/"
        li "http://book.realworldhaskell.org/"

The main application code is now simplified as shown below:

-- template-file.hs

{-# LANGUAGE OverloadedStrings #-}

import qualified Haskell
import Web.Scotty as W
import Text.Blaze.Html
import Text.Blaze.Html.Renderer.Text

blaze :: Text.Blaze.Html.Html -> ActionM ()
blaze = W.html . renderHtml

main :: IO ()
main = scotty 3000 $ do
  get "/" $ do
    blaze Haskell.render

You need to place both the source files (Haskell.hs and template-file.hs) in the same top-level directory, and you can then compile the template-file.hs file that will also compile the dependency Haskell.hs source file as shown below:

$ ghc --make template-file.hs 

[1 of 2] Compiling Haskell          ( Haskell.hs, Haskell.o )
[2 of 2] Compiling Main             ( template-file.hs, template-file.o )
Linking template-file ...

You can now run the server as follows:

$ ./template-file 
Setting phasers to stun... (port 3000) (ctrl-c to quit)

Executing template-file produces the same output as in the case of the template.hs example.

$ curl localhost:3000
<html><body><h1>Haskell list</h1><ul><li>http://haskell.org</li><li>http://learnyouahaskell.com/</li><li>http://book.realworldhaskell.org/</li></ul></body></html>

You can refer the Scotty wiki page at https://github.com/scotty-web/scotty/wiki for more information.

The clay package is a CSS preprocessor similar to LESS and Sass. You can install it using the following Cabal command:

    $ cabal install clay

Let us consider a simple CSS example to generate a list of fonts to be used in the body section of a HTML page. The corresponding Clay Haskell embedded DSL looks like the following:

-- clay-simple.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss exampleStylesheet

exampleStylesheet :: Css
exampleStylesheet = body ? fontFamily ["Baskerville", "Georgia", "Garamond", "Times"] [serif]

You can compile the above code as follows:

$ ghc --make clay-simple.hs 
[1 of 1] Compiling Main             ( clay-simple.hs, clay-simple.o )
Linking clay-simple ...

You can then execute clay-simple to generate the required CSS output as shown below:

$ ./clay-simple

body
{
  font-family : "Baskerville","Georgia","Garamond","Times", serif;
}

/* Generated with Clay, http://fvisser.nl/clay */

A more comprehensive example is shown below for the HTML pre tag:

-- clay-pre.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss $
  pre ?
    do border dotted (pt 1) black
       whiteSpace (other "pre")
       fontSize (other "8pt")
       overflow (other "auto")
       padding (em 20) (em 0) (em 20) (em 0)

You can compile the above clay-pre.hs file as shown below:

$ ghc --make clay-pre.hs 
[1 of 1] Compiling Main             ( clay-pre.hs, clay-pre.o )
Linking clay-pre ...

Executing the above complied clay-pre binary produces the following output:

$ ./clay-pre 

pre
{
  border      : dotted 1pt rgb(0,0,0);
  white-space : pre;
  font-size   : 8pt;
  overflow    : auto;
  padding     : 20em 0em 20em 0em;
}

/* Generated with Clay, http://fvisser.nl/clay */

You can also add custom values using the Other type class or the fallback operator `-:’ to explicitly specify values. For example:

-- clay-custom.hs

{-# LANGUAGE OverloadedStrings #-}

import Clay

main :: IO ()
main = putCss $
  body ?
       do fontSize (other "11pt !important")
          "border" -: "0"

Compiling and executing the above code produces the following output:

$ ghc --make clay-custom.hs 
[1 of 1] Compiling Main             ( clay-custom.hs, clay-custom.o )
Linking clay-custom ...

$ ./clay-custom 

body
{
  font-size : 11pt !important;
  border    : 0;
}

/* Generated with Clay, http://fvisser.nl/clay */

You can explore more of clay from the official project homepage http://fvisser.nl/clay/.

A number of good books are available for further learning. I recommend the following books available online and in print:

  1. Bryan O’Sullivan, Don Stewart, and John Goerzen. (December 1, 2008). Real World Haskell (http://book.realworldhaskell.org/). O’Reilly.

  2. Miran Lipovaca. (April 21, 2011). Learn You a Haskell for Great Good! A Beginner’s Guide (http://learnyouahaskell.com/). No Starch Press.

The https://www.haskell.org website also has plenty of useful resources. You can also join the haskell-cafe@haskell.org and beginners@haskell.org mailing lists ( https://wiki.haskell.org/Mailing_lists ) for discussions. The folks in the #haskell channel on irc.freenode.net are also very helpful.

I hope you enjoyed learning Haskell through this series, as much as I did creating them. Please feel free to write to me (author at shakthimaan dot com) with any feedback or suggestions.

Happy Hacking!

[Published in Open Source For You (OSFY) magazine, May 2015 edition.]

In this article we shall explore network programming in Haskell.

Let us begin with a simple TCP (Transmission Control Protocol) client and server example. The network package provides a high-level interface for communication. You can install the same in Fedora, for example, using the following command:

$ sudo yum install ghc-network

Consider the following simple TCP client code:

-- tcp-client.hs

import Network
import System.IO

main :: IO ()
main = withSocketsDo $ do
         handle <- connectTo "localhost" (PortNumber 3001)
         hPutStr handle "Hello, world!"
         hClose handle

After importing the required libraries, the main function connects to a localhost server running on port 3001, sends a string “Hello, world!”, and closes the connection.

The connectTo function defined in the Network module accepts a hostname, port number and returns a handle that can be used to transfer or receive data.

The type signatures of the withSocketsdo and connectTo functions are as under:

ghci> :t withSocketsDo
withSocketsDo :: IO a -> IO a

ghci> :t connectTo
connectTo :: HostName -> PortID -> IO GHC.IO.Handle.Types.Handle

The simple TCP server code is illustrated below:

-- tcp-server.hs

import Network
import System.IO

main :: IO ()
main = withSocketsDo $ do
         sock <- listenOn $ PortNumber 3001
         putStrLn "Starting server ..."
         handleConnections sock

handleConnections :: Socket -> IO ()
handleConnections sock = do
  (handle, host, port) <- accept sock
  output <- hGetLine handle
  putStrLn output
  handleConnections sock

The main function starts a server on port 3001 and transfers the socket handler to a handleConnections function. It accepts any connection requests, reads the data, prints it to the server log, and waits for more clients.

Firstly, you need to compile the tcp-server.hs and tcp-client.hs files using GHC:

$ ghc --make tcp-server.hs
[1 of 1] Compiling Main             ( tcp-server.hs, tcp-server.o )
Linking tcp-server ...

$ ghc --make tcp-client.hs 
[1 of 1] Compiling Main             ( tcp-client.hs, tcp-client.o )
Linking tcp-client ...

You can now start the TCP server in a terminal:

$ ./tcp-server 
Starting server ...

You can then run the TCP client in another terminal:

$ ./tcp-client

You will now observe the “Hello, world” message printed in the terminal where the server is running:

$ ./tcp-server 
Starting server ...
Hello, world!

The Network.Socket package exposes more low-level socket functionality for Haskell and can be used if you need finer access and control. For example, consider the following UDP (User Datagram Protocol) client code:

-- udp-client.hs

import Network.Socket

main :: IO ()
main = withSocketsDo $ do
         (server:_) <- getAddrInfo Nothing (Just "localhost") (Just "3000")
         s <- socket (addrFamily server) Datagram defaultProtocol
         connect s (addrAddress server)
         send s "Hello, world!"
         sClose s

The getAddrInfo function resolves a host or service name to a network address. A UDP client connection is then requested for the server address, a message is sent, and the connection is closed. The type signatures of getAddrInfo, addrFamily, and addrAddress are given below:

ghci> :t getAddrInfo
getAddrInfo
  :: Maybe AddrInfo
     -> Maybe HostName -> Maybe ServiceName -> IO [AddrInfo]

ghci> :t addrFamily
addrFamily :: AddrInfo -> Family

ghci> :t addrAddress
addrAddress :: AddrInfo -> SockAddr

The corresponding UDP server code is as follows:

-- udp-server.hs

import Network.Socket

main :: IO ()
main = withSocketsDo $ do
         (server:_) <- getAddrInfo Nothing (Just "localhost") (Just "3000")
         s <- socket (addrFamily server) Datagram defaultProtocol
         bindSocket s (addrAddress server) >> return s
         putStrLn "Server started ..."
         handleConnections s

handleConnections :: Socket -> IO ()
handleConnections conn = do
  (text, _, _) <- recvFrom conn 1024
  putStrLn text
  handleConnections conn

The UDP server binds to localhost and starts to listen on port 3000. When a client connects, it reads a maximum of 1024 bytes of data, prints it to stdout, and waits to accept more connections. You can compile the udp-server.hs and udp-client.hs files using the following commands:

$ ghc --make udp-server.hs 
[1 of 1] Compiling Main             ( udp-server.hs, udp-server.o )
Linking udp-server ...

$ ghc --make udp-client.hs 
[1 of 1] Compiling Main             ( udp-client.hs, udp-client.o )
Linking udp-client ...

You can start the UDP server in one terminal:

$ ./udp-server 
Server started ...

You can then run the UDP client in another terminal:

$ ./tcp-client

You will now see the “Hello, world!” message printed in the terminal where the server is running:

$ ./udp-server 
Server started ...
Hello, world!

The network-uri module has many useful URI (Uniform Resource Identifier) parsing and test functions. You can install the same on Fedora using the following command:

$ cabal install network-uri

The parseURI function takes a string and attempts to convert it into a URI. It returns ‘Nothing’ if the input is not a valid URI, and returns the URI, otherwise. For example:

ghci> :m + Network.URI

ghci> parseURI "http://www.shakthimaan.com"
Just http://www.shakthimaan.com

ghci> parseURI "shakthimaan.com"
Nothing

The type signature of the parseURI function is given below:

ghci> :t parseURI
parseURI :: String -> Maybe URI

A number of functions are available for testing the input URI as illustrated in the following examples:

ghci> isURI "shakthimaan.com"
False

ghci> isURI "http://www.shakthimaan.com"
True

ghci> isRelativeReference "http://shakthimaan.com"
False

ghci> isRelativeReference "../about.html"
True

ghci> isAbsoluteURI "http://www.shakthimaan.com"
True

ghci> isAbsoluteURI "shakthimaan.com"
False

ghci> isIPv4address "192.168.100.2"
True

ghci> isIPv6address "2001:0db8:0a0b:12f0:0000:0000:0000:0001"
True

ghci> isIPv6address "192.168.100.2"
False

ghci> isIPv4address "2001:0db8:0a0b:12f0:0000:0000:0000:0001"
False

The type signatures of the above functions are as follows:

ghci> :t isURI
isURI :: String -> Bool

ghci> :t isRelativeReference
isRelativeReference :: String -> Bool

ghci> :t isAbsoluteURI
isAbsoluteURI :: String -> Bool

ghci> :t isIPv4address
isIPv4address :: String -> Bool

ghci> :t isIPv6address
isIPv6address :: String -> Bool

You can make a GET request for a URL and retrieve its contents. For example:

import Network
import System.IO

main = withSocketsDo $ do
    h <- connectTo "www.shakthimaan.com" (PortNumber 80)
    hSetBuffering h LineBuffering
    hPutStr h "GET / HTTP/1.1\nhost: www.shakthimaan.com\n\n"
    contents <- hGetContents h
    putStrLn contents
    hClose h

You can now compile and execute the above code, and it returns the index.html contents as shown below:

$ ghc --make get-network-uri.hs
[1 of 1] Compiling Main             ( get-network-uri.hs, get-network-uri.o )
Linking get-network-uri ...

$ ./get-network-uri 
HTTP/1.1 200 OK
Date: Sun, 05 Apr 2015 01:37:19 GMT
Server: Apache
Last-Modified: Tue, 08 Jul 2014 04:01:16 GMT
Accept-Ranges: bytes
Content-Length: 4604
Content-Type: text/html
...

You can refer to the network-uri package documentation at https://hackage.haskell.org/package/network-uri-2.6.0.1/docs/Network-URI.html for more detailed information.

The whois Haskell package allows you to query for information about hosting servers and domain names. You can install the package on Ubuntu, for example, using:

$ cabal install whois

The serverFor function returns a whois server that can be queried for obtaining more information regarding an IP or domain name. For example:

ghci> :m + Network.Whois

ghci> serverFor "shakthimaan.com"
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package network-2.6.0.2 ... linking ... done.
Loading package transformers-0.4.3.0 ... linking ... done.
Loading package mtl-2.2.1 ... linking ... done.
Loading package text-1.2.0.4 ... linking ... done.
Loading package parsec-3.1.9 ... linking ... done.
Loading package network-uri-2.6.0.1 ... linking ... done.
Loading package split-0.2.2 ... linking ... done.
Loading package whois-1.2.2 ... linking ... done.

Just (WhoisServer {hostname = "com.whois-servers.net", port = 43, query = "domain "})

You can use the above specific information with the whois1 function to make a DNS (Domain Name System) query:

ghci> whois1 "shakthimaan.com" WhoisServer {hostname = "com.whois-servers.net", port = 43, query = "domain "}
Just "\nWhois Server Version 2.0\n\nDomain names in the .com and .net domains can now be registered\n
...

You can also use the whois function to return information on the server as shown below:

ghci> whois "shakthimaan.com"
Just "\nWhois Server Version 2.0\n\nDomain names in the .com and .net domains can now be registered\n
...

The type signatures of serverFor, whois1 and whois functions are as follows:

ghc> :t serverFor
serverFor :: String -> Maybe WhoisServer

ghci> :t whois1
whois1 :: String -> WhoisServer -> IO (Maybe String)

ghci> :t whois
whois :: String -> IO (Maybe String, Maybe String)

The dns package provides a number of useful functions to make Domain Name System queries, and handle the responses. You can install the same on Ubuntu, for example, using the following commands:

$ sudo apt-get install zlib1g-dev
$ cabal install dns

A simple example of finding the IP addresses for the haskell.org domain is shown below:

ghci> import Network.DNS.Lookup
ghci> import Network.DNS.Resolver

ghci> let hostname = Data.ByteString.Char8.pack "www.haskell.org"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupA resolver hostname
Right [108.162.203.60,108.162.204.60]

The defaultResolvConf is of type ResolvConf and consists of the following default values:

--     * 'resolvInfo' is 'RCFilePath' \"\/etc\/resolv.conf\".
--
--     * 'resolvTimeout' is 3,000,000 micro seconds.
--
--     * 'resolvRetry' is 3.

The makeResolvSeed, and withResolver functions assist in making the actual DNS resolution. The lookupA function obtains all the A records for the DNS entry. Their type signatures are shown below:

ghci> :t makeResolvSeed
makeResolvSeed :: ResolvConf -> IO ResolvSeed

ghci> :t withResolver
withResolver :: ResolvSeed -> (Resolver -> IO a) -> IO a

ghci> :t lookupA
lookupA
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [iproute-1.4.0:Data.IP.Addr.IPv4])

The lookupAAAA function returns all the IPv6 ‘AAAA’ records for the domain. For example:

ghci> withResolver rs $ \resolver -> lookupAAAA resolver hostname
Right [2400:cb00:2048:1::6ca2:cc3c,2400:cb00:2048:1::6ca2:cb3c]

Its type signature is shown below:

lookupAAAA
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [iproute-1.4.0:Data.IP.Addr.IPv6])

The MX records for the hostname can be returned using the lookupMX function. An example for the shakthimaan.com website is as follows:

ghci> import Network.DNS.Lookup
ghci> import Network.DNS.Resolver

ghci> let hostname = Data.ByteString.Char8.pack "www.shakthimaan.com"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupMX resolver hostname
Right [("shakthimaan.com.",0)]

The type signature of the lookupMX function is as under:

ghci> :t lookupMX
lookupMX
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [(dns-1.4.5:Network.DNS.Internal.Domain, Int)])

The nameservers for the domain can be returned using the lookupNS function. For example:

ghci> withResolver rs $ \resolver -> lookupNS resolver hostname
Right ["ns22.webhostfreaks.com.","ns21.webhostfreaks.com."]

The type signature of the lookupNS function is shown below:

ghci> :t lookupNS
lookupNS
  :: Resolver
     -> dns-1.4.5:Network.DNS.Internal.Domain
     -> IO
          (Either
             dns-1.4.5:Network.DNS.Internal.DNSError
             [dns-1.4.5:Network.DNS.Internal.Domain])

You can also return the entire DNS response using the lookupRaw function as illustrated below:

ghci> :m + Network.DNS.Types

ghci> let hostname = Data.ByteString.Char8.pack "www.ubuntu.com"

ghci> rs <- makeResolvSeed defaultResolvConf

ghci> withResolver rs $ \resolver -> lookupRaw resolver hostname A
Right (DNSFormat 
  {header = DNSHeader 
    {identifier = 29504, 
     flags = DNSFlags 
       {qOrR = QR_Response, 
        opcode = OP_STD, 
        authAnswer = False, 
        trunCation = False, 
        recDesired = True, 
        recAvailable = True, 
        rcode = NoErr}, 
     qdCount = 1, 
     anCount = 1, 
     nsCount = 3, 
     arCount = 3}, 
   question = [
     Question 
       {qname = "www.ubuntu.com.", 
        qtype = A}], 
     answer = [
       ResourceRecord 
         {rrname = "www.ubuntu.com.", 
          rrtype = A, 
          rrttl = 61, 
          rdlen = 4, 
          rdata = 91.189.89.103}], 
     authority = [
       ResourceRecord 
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 16, 
          rdata = ns2.canonical.com.},
       ResourceRecord 
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 6, 
          rdata = ns1.canonical.com.},
       ResourceRecord
         {rrname = "ubuntu.com.", 
          rrtype = NS, 
          rrttl = 141593, 
          rdlen = 6, 
          rdata = ns3.canonical.com.}], 
    additional = [
      ResourceRecord 
        {rrname = "ns2.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.95.3},
      ResourceRecord 
        {rrname = "ns3.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.91.139},
      ResourceRecord 
        {rrname = "ns1.canonical.com.", 
         rrtype = A, 
         rrttl = 88683, 
         rdlen = 4, 
         rdata = 91.189.94.173}]})

Please refer the Network.DNS hackage web page https://hackage.haskell.org/package/dns for more information.

[Published in Open Source For You (OSFY) magazine, April 2015 edition.]

In this article we shall explore access to Redis and PostgreSQL databases using Haskell modules. The hackage website at https://hackage.haskell.org/packages/#cat:Database provides a vast number of database packages that you can use, a couple of which will be covered here..

You will need to install the cabal-install tool on Fedora, for example, using the following command:

$ sudo yum install cabal-install
  • Connecting to the Redis database

Let’s use the hedis package to connect to the Redis server. Install the Fedora dependency package alex, and the Redis server as shown below:

$ sudo yum install alex redis

You can then install the hedis package using the following commands:

$ cabal update
$ cabal install hedis

This installs the latest hedis version 0.6.5. You can now start the Redis server on Fedora using the service command:

$ sudo service redis start

You can then test connectivity to the Redis server using the redis-cli command by issuing the PING command as follows:

$ redis-cli

127.0.0.1:6379> PING
PONG

You can also test the same using the hedis package inside the GHCi interpreter as illustrated below:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :m Database.Redis 

ghci> conn <- connect defaultConnectInfo

Loading package array-0.4.0.1 ... linking ... done.
Loading package base-unicode-symbols-0.2.2.4 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package stm-2.4.2 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package hashable-1.1.2.5 ... linking ... done.
Loading package transformers-base-0.4.1 ... linking ... done.
Loading package monad-control-0.3.2.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package BoundedChan-1.0.3.0 ... linking ... done.
Loading package bytestring-lexing-0.4.3.2 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package network-2.6.0.2 ... linking ... done.
Loading package resource-pool-0.2.3.2 ... linking ... done.
Loading package hedis-0.6.5 ... linking ... done.

ghci> runRedis conn ping
Right Pong

It is recommended that you use defaultConnectInfo to connect to the database, and its type is ConnectInfo:

ghci> :t defaultConnectInfo
defaultConnectInfo :: ConnectInfo

The different options that can be used in defaultConnectInfo are as follows:

connectHost           = "localhost"
connectPort           = PortNumber 6379 -- Redis port
connectAuth           = Nothing         -- No authentication
connectDatabase       = 0               -- SELECT database 0
connectMaxConnections = 10              -- Up to 10 connections
connectMaxIdleTime    = 20              -- Keep connection open for 20 seconds

The types of conn, connect, runRedis and ping are given below:

ghci> :t conn
conn :: Connection

ghci> :t connect
connect :: ConnectInfo -> IO Connection

ghci> :t runRedis
runRedis :: Connection -> Redis a -> IO a

ghci> :t ping
ping :: RedisCtx m f => m (f Status)

If the Redis server was not started, and you tried to issue the ping command, the following exception will be automatically thrown by the package:

ghci> runRedis conn ping
*** Exception: connect: does not exist (No route to host)

You can automate the above code snippets into Haskell code with a main function as demonstrated below:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis

main :: IO (Either Reply Status)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn ping

The OverloadedStrings extension allows string literals to be polymorphic for the IsString class. You can compile and run the above code inside GHCi, as follows:

$ ghci ping.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( ping.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
...
Right Pong

The echo Redis command is used to print a message that is passed as an argument to it. The equivalent hedis echo command expects the message to be of type ByteString. For example:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis
import qualified Data.ByteString as B

bytes :: B.ByteString
bytes = "Hello, World" :: B.ByteString

main :: IO (Either Reply B.ByteString)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ echo bytes

Loading the above code in GHCi produces the following output:

ghci> main
Right "Hello, World"

The type signature of the echo function is as follows:

echo
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> m (f Data.ByteString.Internal.ByteString)

You can set a value to a key using the set function in hedis. An example is shown below:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis

main :: IO (Either Reply Status)
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ set "a" "apple"

Loading the above set.hs code in GHCi and testing the same produces the following output:

ghci> :l set.hs
[1 of 1] Compiling Main             ( set.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Right Ok

The type signature of the set function is shown below:

ghci> :t set
set
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> Data.ByteString.Internal.ByteString -> m (f Status)

You can verify the value of the key `a’ from the redis-cli command, and it must return the value “apple”:

127.0.0.1:6379> get a
"apple"

You can also retrieve the value of a key using the get function. For example:

{-# LANGUAGE OverloadedStrings #-}
import Database.Redis
import Control.Monad.IO.Class

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
  runRedis conn $ do
         result <- get "a"
         liftIO $ print result

Executing the above code in GHCi gives the expected result:

ghci> :l get.hs
[1 of 1] Compiling Main             ( get.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Right (Just "apple")

The liftIO function transforms an IO action into a Monad. Its type signature is shown below:

ghci> :t liftIO
liftIO :: MonadIO m => IO a -> m a

The type signature of the get function is as follows:

ghci> :t get
get
  :: RedisCtx m f =>
     Data.ByteString.Internal.ByteString
     -> m (f (Maybe Data.ByteString.Internal.ByteString))

You are encouraged to read the Database.Redis documentation page that contains a comprehensive list of commands and their usage at https://hackage.haskell.org/package/hedis-0.6.5/docs/Database-Redis.html.

  • Accessing the PostgreSQL database

We shall now explore accessing a PostgreSQL database using the postgresql-simple (0.4.10.0) package. You will need to install and configure PostgreSQL for your GNU/Linux distribution. Please follow your distribution documentation to do so. On Fedora, for example, you can install the database server using the following command:

$ sudo yum install postgresql-server postgresql-contrib

You can then start the database server using the following service command:

$ sudo service postgresql start

You can now install the postgresql-simple package using the cabal command:

$ cabal install postgresql-simple

Let us first create a database and a schema using the Postgresql command-line utility psql:

$ psql -U postgres
Password for user postgres: 
psql (9.3.5)
Type "help" for help.

postgres=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 postgres  | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | 
 template0 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
 template1 | postgres | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/postgres          +
           |          |          |             |             | postgres=CTc/postgres
(3 rows)

postgres=# CREATE DATABASE test;
CREATE DATABASE

postgres-# \c test
You are now connected to database "test" as user "postgres".

test=# create schema social;
CREATE SCHEMA

test=# \dn
 public | postgres
 social | postgres

We can then create a users’ table with an id, first name and last name using the postgresql-simple package.

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }

  execute conn "create table social.users (id INT, fname VARCHAR(80), lname VARCHAR(80))" ()

  close conn

Loading the above code in GHCi creates the table social.users as shown below:

$ ghci create.hs 

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( create.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package blaze-builder-0.3.1.1 ... linking ... done.
Loading package dlist-0.5 ... linking ... done.
Loading package hashable-1.1.2.5 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package syb-0.4.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package unordered-containers-0.2.3.0 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package aeson-0.6.2.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package scientific-0.2.0.2 ... linking ... done.
Loading package case-insensitive-1.0.0.1 ... linking ... done.
Loading package blaze-textual-0.2.0.8 ... linking ... done.
Loading package postgresql-libpq-0.9.0.2 ... linking ... done.
Loading package binary-0.7.4.0 ... linking ... done.
Loading package cereal-0.3.5.2 ... linking ... done.
Loading package entropy-0.2.2.1 ... linking ... done.
Loading package tagged-0.6 ... linking ... done.
Loading package crypto-api-0.11 ... linking ... done.
Loading package cryptohash-0.9.0 ... linking ... done.
Loading package network-info-0.2.0.5 ... linking ... done.
Loading package uuid-1.3.8 ... linking ... done.
Loading package postgresql-simple-0.4.10.0 ... linking ... done.

You can verify the created table from the psql prompt:

test=# \d social.users
 id     | integer               | 
 fname  | character varying(80) | 
 lname  | character varying(80) | 

You can also list the databases in the PostgreSQL server using the query_ function as illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  databases <- query_ conn "SELECT datname FROM pg_database"
  print (databases :: [Only String])

  close conn

Executing the above code in GHCi produces the following output:

$ ghci show.hs 
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( show.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
[Only {fromOnly = "template1"},Only {fromOnly = "template0"},Only {fromOnly = "postgres"},Only {fromOnly = "test"}]

You can now insert a record into the databaes using the execute function:

execute conn "insert into social.users (id, fname, lname) values (?, ?, ?)" ["1" :: String, "Edwin" :: String, "Brady" :: String]

After executing the above code, you can verify the database entry from the psql prompt:

test=# select * from social.users;
 id | fname | lname 
----+-------+-------
  1 | Edwin | Brady
(1 row)

You can also do batch inserts using the executeMany function. For example:

executeMany conn "insert into social.users (id, fname, lname) values (?, ?, ?)" [("2" :: String, "Simon" :: String, "Marlow" :: String), ("3" :: String, "Ulf" :: String, "Norell" :: String)]

After running the above code, you can check the newly added rows in the database from the psql command-line tool:

test=# select * from social.users;
 id | fname | lname  
----+-------+--------
  1 | Edwin | Brady
  2 | Simon | Marlow
  3 | Ulf   | Norell
(3 rows)

You can also change a record entry using the UPDATE statement as shown below:

execute conn "update social.users SET lname = 'Peyton Jones' where fname = 'Simon'" ()

The corresponding entry is updated as seen from the psql prompt:

test=# select * from social.users;
 id | fname |    lname     
----+-------+--------------
  1 | Edwin | Brady
  3 | Ulf   | Norell
  2 | Simon | Peyton Jones
(3 rows)

It is recommended to catch exceptions when running database commands. Consider the following example, where the number of arguments passed does not match with the expected:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple
import Control.Exception
import GHC.Int

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  result <- try (execute conn "insert into social.users (id, fname, lname) values (?, ?, ?)" ["4" :: String, "Laurel" :: String]) :: IO (Either SomeException Int64)
  case result of
      Left ex  -> putStrLn $ "Caught exception: " ++ show ex
      Right val -> putStrLn $ "The answer was: " ++ show val
  close conn

The error is observed when the main function is executed as shown below:

ghci> main
Caught exception: FormatError {fmtMessage = "3 '?' characters, but 2 parameters", fmtQuery = "insert into social.users (id, fname, lname) values (?, ?, ?)", fmtParams = ["4","Laurel"]}

You can also retrieve multiple records from the database and use the results using a map function. An example is illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.PostgreSQL.Simple
import Control.Monad
import Data.Text as Text

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "postgres"
    , connectPassword = "postgres123"
    , connectDatabase = "test"
    }
  users <- query_ conn "SELECT fname, lname FROM social.users"
  forM_ users $ \(fname, lname) ->
      putStrLn $ Text.unpack fname ++ " " ++ Text.unpack lname
  close conn

The output after executing the above code in GHCi returns the actual data:

ghci> main
Edwin Brady
Ulf Norell
Simon Peyton Jones

Please refer to the Database.PostgreSQL.Simple documentation for more examples and usage at https://hackage.haskell.org/package/postgresql-simple-0.4.10.0/docs/Database-PostgreSQL-Simple.html.

[Published in Open Source For You (OSFY) magazine, March 2015 edition.]

In this ninth article on Haskell, I shall cover access to Sqlite, and MySQL databases using Haskell modules. A number of packages are available from the https://hackage.haskell.org/packages/#cat:Database website, but, I will illustrate a few of them with examples.

You first need to install the cabal-install tool on Fedora, for example, using the following command:

$ sudo yum install cabal-install

You can then install HDBC.Sqlite3 using the following command:

$ cabal install HDBC-sqlite3

This installs the latest 2.3.3.0 version from https://hackage.haskell.org/package/HDBC-sqlite3. You can also install the Sqlite3 package on Fedora for testing, as follows:

$ sudo yum install sqlite

To initiate a connection to a database, you can test it out in the GHCi prompt using the connectSqlite3 function, as shown below:

$ ghci

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :module Database.HDBC Database.HDBC.Sqlite3

ghci> conn <- connectSqlite3 "students.db"

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package old-time-1.1.0.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package utf8-string-0.3.7 ... linking ... done.
Loading package mtl-2.1.2 ... linking ... done.
Loading package convertible-1.1.0.0 ... linking ... done.
Loading package HDBC-2.4.0.0 ... linking ... done.
Loading package HDBC-sqlite3-2.3.3.0 ... linking ... done.

The signature of the connectSqlite3 function is as follows:

ghci> :t connectSqlite3
connectSqlite3 :: FilePath -> IO Connection

The type of conn is a Connection.

ghci> :t conn
conn :: Connection

If you already have an existing Sqlite3 database, you can give the full path to the database and connect to it, or else you can now create a table using the Sqlite CREATE TABLE syntax as shown below:

ghci> run conn "CREATE TABLE names (id INTEGER NOT NULL, fname VARCHAR(80), lname VARCHAR(80))" []
0

The type signature of run is as follows:

ghci> :t run
run
  :: IConnection conn => conn -> String -> [SqlValue] -> IO Integer

It takes three arguments as input and performs an IO computation that returns an integer indicating the status of the execution. The first argument to run is the connection, the second argument is the Sqlite command to be executed, and finally is the array of SqlValues that provide a mapping between Haskell values and SQL databases.

Both Haskell and SQL databases have types, and different databases may have different representations of the types. In order to provide a consistent mapping between the two, each HDBC driver implements the relation using SqlValue.

You can now insert a record into the database using the following command:

ghci> run conn "INSERT INTO names (id, fname, lname) VALUES(1, 'Edwin', 'Brady')" []
1

ghci> commit conn

The type signature of commit is given here:

ghci> :t commit
commit :: IConnection conn => conn -> IO ()

It takes a connection and completes the pending IO actions. To read the result from Haskell you can use the quickQuery function from the GHCi prompt, as follows:

ghci> quickQuery conn "SELECT * from names" []
[[SqlByteString "1",SqlByteString "Edwin",SqlByteString "Brady"]]

The type signature of the quickQuery function is as follows:

quickQuery
  :: IConnection conn =>
     conn -> String -> [SqlValue] -> IO [[SqlValue]]

You can also verify the result of the above actions using the sqlite3 executable in the command prompt as illustrated below:

$ sqlite3 students.db 

SQLite version 3.8.4.3 2014-04-03 16:53:12
Enter ".help" for usage hints.

sqlite> .schema
CREATE TABLE names (id INTEGER NOT NULL, fname VARCHAR(80), lname VARCHAR(80));

sqlite> select * from names;
1|Edwin|Brady

You can also do batch processing for inserts by preparing the statements and executing them:

ghci> batch <- prepare conn "INSERT INTO names VALUES (?, ?, ?)"

ghci> execute batch [toSql (2 :: Int), toSql "Simon", toSql "Marlow"]
1

ghci> execute batch [toSql (3 :: Int), toSql "Ulf", toSql "Norell"]
1

ghci> commit conn

The type signatures of the prepare and execute functions are given below:

ghci> :t prepare
prepare :: IConnection conn => conn -> String -> IO Statement

ghci> :t execute
execute :: Statement -> [SqlValue] -> IO Integer

You can once again check the records in the database using the quickQuery function:

ghci> quickQuery' conn "SELECT * from names" []

[[SqlByteString "1",SqlByteString "Edwin",SqlByteString "Brady"],[SqlByteString "2",SqlByteString "Simon",SqlByteString "Marlow"],[SqlByteString "3",SqlByteString "Ulf",SqlByteString "Norell"]]

You can also run an update query to the database. For example:

ghci> run conn "UPDATE names set lname = 'Peyton Jones' WHERE fname = 'Simon'" []
1

ghci> commit conn

Verifying the output from the Sqlite3 command prompt, you get:

sqlite> select * from names;

1|Edwin|Brady
2|Simon|Peyton Jones
3|Ulf|Norell

The HDBC driver provides many functions to retrieve information regarding the database and the drivers. A few examples are illustrated below:

ghci> hdbcDriverName conn
"sqlite3"

ghci> hdbcClientVer conn
"3.8.4.3"

ghci> dbTransactionSupport conn
True

ghci> getTables conn
["names"]

ghci> describeTable conn "names"

[("id",SqlColDesc {colType = SqlIntegerT, colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing}),("fname",SqlColDesc {colType = SqlUnknownT "varchar(80)", colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing}),("lname",SqlColDesc {colType = SqlUnknownT "varchar(80)", colSize = Nothing, colOctetLength = Nothing, colDecDigits = Nothing, colNullable = Nothing})]

It is considered good practice to use handleSqlError before running any HDBC commands to catch errors that may arise during the database transactions. For example:

ghci> handleSqlError $ quickQuery conn "SELECT * from namesaaa" []

*** Exception: user error (SQL error: SqlError {seState = "", seNativeError = 1, seErrorMsg = "prepare 23: SELECT * from namesaaa: no such table: namesaaa"})

To disconnect from the database, you can use the disconnect function provided by HDBC as shown below:

ghci> disconnect conn

Let’s now look at how to access a MySQL database using the mysql-simple package. You will also need to create or grant privileges for an existing user to use the MySQL database server. Please follow your GNU/Linux distribution manual on how to install and configure a MySQL server. On Fedora, for example, you must have mysql and mysql-server installed:

$ sudo yum install community-mysql community-mysql-server

You can install the mysql-simple Haskell package using:

$ cabal install mysql-simple

Create a test database using the mysql command line tool as shown below:

$ mysql -u user -p

Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.38-log MySQL Community Server (GPL)

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database test;
Query OK, 1 row affected (0.03 sec)

Let us create a users’ table that has an id as well as first name and last name fields, using the mysql-simple package, as follows:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }

  execute conn "create table users (id INT, fname VARCHAR(80), lname VARCHAR(80))" ()

  close conn

The OverloadedStrings extension allows string literals to be polymorphic for the IsString class. The defaultConnectInfo is of type ConnectInfo:

ghci> :t defaultConnectInfo
defaultConnectInfo :: ConnectInfo

ConnectInfo can take many parameters to describe the connectivity to the MySQL server. For example:

connectInfo :: ConnectInfo
connectInfo = ConnectInfo { connectHost = "localhost",
                            connectPort = 3306,
                            connectUser = "user",
                        connectPassword = "password",
                        connectDatabase = "test",
                         connectOptions = [],
                            connectPath = "",
                             connectSSL = Nothing }

The above code to create a table can be compiled directly in GHCi and the main function can be executed as given below:

$ ghci create.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( create.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package text-0.11.3.1 ... linking ... done.
Loading package blaze-builder-0.3.1.1 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package vector-0.10.0.1 ... linking ... done.
Loading package pcre-light-0.4 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package attoparsec-0.10.4.0 ... linking ... done.
Loading package blaze-textual-0.2.0.8 ... linking ... done.
Loading package base16-bytestring-0.1.1.6 ... linking ... done.
Loading package mysql-0.1.1.7 ... linking ... done.
Loading package mysql-simple-0.2.2.4 ... linking ... done.

You can check with the mysql command line utility for the created table:

mysql> use test;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> desc users;
+-------+-------------+------+-----+---------+-------+
| Field | Type        | Null | Key | Default | Extra |
+-------+-------------+------+-----+---------+-------+
| id    | int(11)     | YES  |     | NULL    |       |
| fname | varchar(80) | YES  |     | NULL    |       |
| lname | varchar(80) | YES  |     | NULL    |       |
+-------+-------------+------+-----+---------+-------+
3 rows in set (0.00 sec)

You can now list the databases available in the MySQL server using the query_ function as illustrated below:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  databases <- query_ conn "SHOW databases"
  print (databases :: [Only String])

  close conn

You can compile the above code directly with GHCi and execute the main function, as follows:

 $ ghci show-databases.hs

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( show-databases.hs, interpreted )
Ok, modules loaded: Main.

ghci> main

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
...
[Only {fromOnly = "information_schema"},Only {fromOnly = "mysql"},Only {fromOnly = "performance_schema"},Only {fromOnly = "test"}]

You can try inserting a record into the database using the execute function:

execute conn "insert into users (id, fname, lname) values (?, ?, ?)" ["1" :: String, "Edwin" :: String, "Brady" :: String]

After running the code, you can check the database entry using the mysql client program as shown below:

mysql> select * from users;

+------+-------+-------+
| id   | fname | lname |
+------+-------+-------+
|    1 | Edwin | Brady |
+------+-------+-------+
1 row in set (0.00 sec)

You can also do batch inserts using the executeMany function. For example:

executeMany conn "insert into users (id, fname, lname) values (?, ?, ?)" [("2" :: String, "Simon" :: String, "Marlow" :: String), ("3" :: String, "Ulf" :: String, "Norell" :: String)]

You can verify the execution of the code from the mysql utility:

mysql> select * from users;
+------+-------+--------+
| id   | fname | lname  |
+------+-------+--------+
|    1 | Edwin | Brady  |
|    2 | Simon | Marlow |
|    3 | Ulf   | Norell |
+------+-------+--------+
3 rows in set (0.01 sec)

You can change a record entry using the UPDATE MySQL command:

execute conn "update users SET lname = 'Peyton Jones' where fname = 'Simon'" ()

Executing the code in GHCi, and checking the results with the mysql prompt gives the following changed output:

mysql> select * from users;
+------+-------+--------------+
| id   | fname | lname        |
+------+-------+--------------+
|    1 | Edwin | Brady        |
|    2 | Simon | Peyton Jones |
|    3 | Ulf   | Norell       |
+------+-------+--------------+
3 rows in set (0.00 sec)

It is important to catch any exceptions that may arise on executing the database commands. Consider the following example, where the number of arguments passed does not match with the expected:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple
import Control.Exception
import GHC.Int

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  result <- try (execute conn "insert into users (id, fname, lname) values (?, ?, ?)" ["4" :: String, "Laurel" :: String]) :: IO (Either SomeException Int64)
  case result of
      Left ex  -> putStrLn $ "Caught exception: " ++ show ex
      Right val -> putStrLn $ "The answer was: " ++ show val
  close conn

The error is caught when the main function is executed inside GHCi:

ghci> main
Caught exception: FormatError {fmtMessage = "3 '?' characters, but 2 parameters", fmtQuery = "insert into users (id, fname, lname) values (?, ?, ?)", fmtParams = ["4","Laurel"]}

You can also map through the results returned from the database and use them for your needs. The following is an illustration of the same:

{-# LANGUAGE OverloadedStrings #-}

import Database.MySQL.Simple
import Control.Monad
import Data.Text as Text

main :: IO ()
main = do
  conn <- connect defaultConnectInfo
    { connectUser = "user"
    , connectPassword = "password"
    , connectDatabase = "test"
    }
  users <- query_ conn "SELECT fname, lname FROM users"
  forM_ users $ \(fname, lname) ->
      putStrLn $ Text.unpack fname ++ " " ++ Text.unpack lname
  close conn

The resultant output when executing the main function in GHCi is given below:

ghci> main

Edwin Brady
Simon Peyton Jones
Ulf Norell

You are encouraged to read Database.MySQL.Simple documentation from https://hackage.haskell.org/package/mysql-simple-0.2.2.4/docs/Database-MySQL-Simple.html for more information.

[Published in Open Source For You (OSFY) magazine, February 2015 edition.]

Let’s take a look at the property-based testing of Haskell programs and at the Cabal tool, which is used to build and manage Haskell packages and applications.

One of the main features of testing in Haskell is property-based testing. The type system allows you to infer and derive types, and also helps in auto-generating test cases. QuickCheck is a popular property-based testing library for Haskell. If your program is pure, you can write tests to ascertain the properties and invariants of your programs, and the tests can be auto-generated and executed.

You can install QuickCheck on Fedora, for example, by using the following command:

$ sudo yum install ghc-QuickCheck-devel

Consider a simple function to add two integers:

mySum :: Int -> Int -> Int
mySum a b = a + b

We can ascertain the property of the function that ‘a + b’ is the same as ‘b + a’ using the QuickCheck library. You must first define the invariant in a function as shown below:

prop_mySum a b = mySum a b == mySum b a

You can test the code directly in the GHCi prompt, using the following command:

$ ghci sum.hs 
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( sum.hs, interpreted )
Ok, modules loaded: Main.

ghci> prop_mySum 2 3
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package QuickCheck-2.6 ... linking ... done.
True

You can also invoke the quickCheck function in a main function, as shown below:

import Test.QuickCheck

mySum :: Int -> Int -> Int
mySum a b = a + b

prop_mySum :: Int -> Int -> Bool
prop_mySum a b = mySum a b == mySum b a

main :: IO ()
main = quickCheck prop_mySum

Compiling and executing the above code produces the following output:

$ ghc --make sum.hs
[1 of 1] Compiling Main             ( sum.hs, sum.o )
Linking sum ...

$ ./sum 
+++ OK, passed 100 tests.

You can also dump the input that was generated for the various test cases using the verboseCheck function, as shown below:

main :: IO ()
main = verboseCheck prop_mySum

Executing the above code with the updated main function will yield 100 input test cases that were generated in runtime.

ghci> main
Passed:
0
0
Passed:
-1
1
Passed:
64
-44
Passed:
-2159
2134
Passed:
-927480859
61832343
...

The head function in Haskell expects to receive a non-empty list. You can write a headExists function to check if the head exists for a list of integers, as shown below:

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

You can load the above code in GHCi and test it out, as follows:

ghci> headExists []
False

ghci> headExists [1, 2, 3]
True

Let’s assume that, by mistake, you wrote an incorrect property-based test where the headExists function will always return ‘False’, ignoring the ‘otherwise’ case.

import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

prop_headExists :: [Int] -> Bool
prop_headExists emptyList = headExists emptyList == False

main :: IO ()
main = quickCheck prop_headExists

Testing the code produces the following output:

$ ghc --make head.hs 
[1 of 1] Compiling Main             ( head.hs, head.o )
Linking head ...

$ ./head 
*** Failed! Falsifiable (after 3 tests):             
[0]

The QuickCheck library generated test cases for different [Int] types and it returned a failure after the third test, for which the input was [0]. Clearly, the ‘headExists [0]’ computation will return ‘True’ and not ‘False’.

The way we defined the property is incorrect. We know that if the list is empty, then its length is zero. We can write a helper function lengthZero for the above, as follows:

lengthZero :: [Int] -> Bool
lengthZero list
    | length list == 0 = True
    | otherwise = False

We can then use this function to assert that for any Integer list, if headExists returns ‘False’ then the lengthZero function must return ‘True’. The complete code is shown below:

import Data.List
import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

lengthZero :: [Int] -> Bool
lengthZero list
    | length list == 0 = True
    | otherwise = False

prop_headExists :: [Int] -> Bool
prop_headExists list = headExists list == not (lengthZero list)

main :: IO ()
main = quickCheck prop_headExists

Executing the code produces the required output:

$ ghc --make head.hs 
[1 of 1] Compiling Main             ( head.hs, head.o )
Linking head ...

$ ./head 
+++ OK, passed 100 tests.

We can also re-write the above code based on conditional properties. The property that the headExists function will return ‘True’ only for non-empty lists can be defined as a constraint. The notation syntax is condition ==> property. In our example, if the condition that the list is non-empty is ‘True’, then the property that the headExists function for the list must return is ‘True’. Also, when the list is empty, the headExists function must return ‘False’. These two conditions can be written as follows:

import Data.List
import Test.QuickCheck

headExists :: [Int] -> Bool
headExists list
    | null list    = False
    | otherwise = True

prop_headExists :: [Int] -> Property
prop_headExists list = length list > 0 ==> headExists list == True

prop_emptyList :: [Int] -> Property
prop_emptyList list = length list == 0 ==> headExists list == False

main :: IO ()
main = do
     quickCheck prop_headExists
     quickCheck prop_emptyList

Testing the code produces the following output:

$ ghc --make cond.hs 
[1 of 1] Compiling Main             ( cond.hs, cond.o )
Linking cond ...

 $ ./cond 
+++ OK, passed 100 tests.
*** Gave up! Passed only 38 tests.

These tests can be integrated with Hspec or HUnit for a more verbose output.

Cabal

Cabal is a software tool that is used to describe a Haskell application, list its dependencies, and provide a manifestation to distribute the source and binaries. It is not to be confused with a distribution package manager like RPM or the Debian package management system. You can install Cabal using your distribution package manager. On Fedora, for example, you can use the following command:

$ sudo yum install cabal-install

Haskell software programs are available in hackage.haskell.org, and each project has a .cabal file. Let us take an example of the HSH-2.1.2 package at http://hackage.haskell.org/package/HSH which allows you to use shell commands and expressions within Haskell programs. You can download HSH-2.1.2.tar.gz and extract it using:

$ tar xzvf HSH-2.1.2.tar.gz 

HSH-2.1.2/
HSH-2.1.2/COPYING
HSH-2.1.2/HSH.cabal
HSH-2.1.2/testsrc/
HSH-2.1.2/testsrc/runtests.hs
HSH-2.1.2/HSH.hs
HSH-2.1.2/HSH/
HSH-2.1.2/HSH/Command.hs
HSH-2.1.2/HSH/ShellEquivs.hs
HSH-2.1.2/HSH/Channel.hs
HSH-2.1.2/COPYRIGHT
HSH-2.1.2/Setup.lhs

The .cabal file has various fields that describe the Haskell application. The contents of the HSH.cabal for version 2.1.2 are given below:

Name: HSH
Version: 2.1.3
License: LGPL
Maintainer: John Goerzen <jgoerzen@complete.org>
Author: John Goerzen
Stability: Beta
Copyright: Copyright (c) 2006-2014 John Goerzen
Category: system
license-file: COPYRIGHT
extra-source-files: COPYING
homepage: http://software.complete.org/hsh
Synopsis: Library to mix shell scripting with Haskell programs
Description: HSH is designed to let you mix and match shell expressions with
 Haskell programs. With HSH, it is possible to easily run shell
 commands, capture their output or provide their input, and pipe them
 to and from other shell commands and arbitrary Haskell functions at will.
  Category: System

Cabal-Version: >=1.2.3
Build-type: Simple

flag buildtests
  description: Build the executable to run unit tests
  default: False

library
  Exposed-Modules: HSH, HSH.Command, HSH.ShellEquivs, HSH.Channel
  Extensions: ExistentialQuantification, OverlappingInstances,
    UndecidableInstances, FlexibleContexts, CPP
  Build-Depends: base >= 4 && < 5, mtl, process, regex-compat, MissingH>=1.0.0,
    hslogger, filepath, regex-base, regex-posix, directory,
    bytestring
  if !os(windows)
    Build-Depends: unix
  GHC-Options: -O2 -threaded -Wall

Executable runtests
  if flag(buildtests)
    Buildable: True
    Build-Depends: base >= 4 && < 5, mtl, process, regex-compat,
      MissingH>=1.0.0,
      hslogger, filepath, regex-base, regex-posix, directory,
      bytestring, HUnit, testpack
    if !os(windows)
      Build-Depends: unix
  else
    Buildable: False
  Main-Is: runtests.hs
  HS-Source-Dirs: testsrc, .
  Extensions: ExistentialQuantification, OverlappingInstances,
    UndecidableInstances, FlexibleContexts, CPP
  GHC-Options: -O2 -threaded

Enter the HSH-2.1.2 directory and configure the project using the cabal configure command as shown below:

$ cd HSH-2.1.2

$ cabal configure
Resolving dependencies...
Configuring HSH-2.1.2...

You can then compile the project sources using the cabal build step:

$ cabal build

Building HSH-2.1.2...
Preprocessing library HSH-2.1.2...
[1 of 4] Compiling HSH.Channel      ( HSH/Channel.hs, dist/build/HSH/Channel.o )
...
[2 of 4] Compiling HSH.Command      ( HSH/Command.hs, dist/build/HSH/Command.o )
...
[3 of 4] Compiling HSH.ShellEquivs  ( HSH/ShellEquivs.hs, dist/build/HSH/ShellEquivs.o )
...
[4 of 4] Compiling HSH              ( HSH.hs, dist/build/HSH.o )
In-place registering HSH-2.1.2...

You can install the built library files using the cabal install command. By default, it installs to ~/.cabal folder as shown below:

$ cabal install

Resolving dependencies...
Configuring HSH-2.1.2...
Building HSH-2.1.2...
Preprocessing library HSH-2.1.2...
In-place registering HSH-2.1.2...
Installing library in /home/guest/.cabal/lib/HSH-2.1.2/ghc-7.6.3
Registering HSH-2.1.2...
Installed HSH-2.1.2

You can also generate HTML documentation for the source code using the cabal haddock option. The HTML files can also be made available at hackage.haskell.org.

$ cabal haddock

Running Haddock for HSH-2.1.2...
Preprocessing library HSH-2.1.2...
Warning: The documentation for the following packages are not installed. No
links will be generated to these packages: MissingH-1.3.0.1, rts-1.0,
hslogger-1.2.6, network-2.6.0.2
Haddock coverage:
...
Documentation created: dist/doc/html/HSH/index.html

If you make changes to the sources and wish to generate a new release, you can update the Version field in the HSH.cabal file.

Version: 2.1.3

In order to make a new tarball, use the cabal sdist command:

$ cabal sdist

Distribution quality warnings:
...
Building source dist for HSH-2.1.3...
Preprocessing library HSH-2.1.3...
Source tarball created: dist/HSH-2.1.3.tar.gz

To test the installed application, you can run GHCi from a directory other than the HSH-2.1.2 sources directory. For example:

$ ghci
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.

ghci> :m + HSH

ghci HSH> runIO "date"
Sun Jan  4 14:22:37 IST 2015

You should not run GHCi from the sources directory, since it will find the module in it and try to use it instead of the installed modules in ~/.cabal folder.

You can also test the installation by writing a program:

import HSH.Command

main :: IO ()
main = do
  runIO "date"

You can compile and execute the above as shown below:

$ ghc --make test.hs 
[1 of 1] Compiling Main             ( test.hs, test.o )
Linking test ...

$ ./test 
Sun Jan  4 14:25:19 IST 2015

You are encouraged to read the Cabal guide at https://www.haskell.org/cabal/ for information on specific fields and their options.

I attended PyCon India 2015 at NIMHANS Convention Centre, Bengaluru, India between October 2-4, 2015.

PyCon India 2015 stage

Day 1

I took an early train from Chennai to reach Bengaluru, and headed straight to the venue. The workshops were scheduled on the first day. After completing the registration formalities, I went to observe the DevSprint. I met a number of dgplug folks with whom I have been interacting online on #dgplug (irc.freenode.net).

After lunch, I attended Prof. Ronojoy Adhikari’s workshop on “Reasoning under uncertainity with Python”. Dorai Thodla began the workshop with an introduction on “Data Science” and its use and applications in the industry. Prof. Ronojoy then spoke on “Data Science: Theory” and “Probability Theory for Data Science”. We already use Boolean logic in our programming. We also have probability concepts implemented in Python. He then proceeded to demonstrate “Probabilistic Programming in Python” using Lea. He had presented a short preview of this talk at the Chennaipy August 2015 meet-up. Prof. Ronojoy showed very interesting Lea one-liners with IPython Notebook. A sample is provided below:

In [1]: from lea import *

In [2]: faircoin = Lea.fromVals('Head', 'Tail')

In [3]: faircoin.random(10)

Out[3]: ('Tail',
         'Tail',
         'Tail',
         'Head',
         'Head',
         'Tail',
         'Tail',
         'Head',
         'Tail',
         'Head')

In [4]: die = Lea.fromVals(1, 2, 3, 4, 5, 6)

In [5]: die1 = die.clone()
        die2 = die.clone()

In [6]: dice = die1 + die2

In [7]: dice

OUt[10]: 2 : 1/36
         3 : 2/36
         4 : 3/36
         5 : 4/36
         6 : 5/36
         7 : 6/36
         8 : 5/36
         9 : 4/36
        10 : 3/36
        11 : 2/36
        12 : 1/36

I had installed the Anaconda distribution and Lea for the workshop, but, couldn’t get Pomegranate to compile. Later, I worked with Jacob Schreiber to get it built on Ubuntu 14.10.

Prof. Ronojoy then explained Bayesian networks, and how the probabilities can be fed to compute the probability for a causal-effect relationship. He demonstrated numerous examples with live coding. His work is part of the Value from Data initiative. Prof. Ronojoy and Dorai’s “Bayesian Inference and Data Analytics” video lectures are available in YouTube.

Day 2

After collecting my speaker kit, I attended the “KG award” presentation, which was given to Vijay Kumar, followed by the key note by Dr. Ajit Kumar. Vijay Kumar is doing an excellent job in organizing the monthly Chennaipy meet-ups. It was also good to meet Dr. Ajit Kumar after many years, having known his work and involvement with the Phoenix project.

During the tea break, I took the time to test my laptop with the projector in auditorium-1, where I was scheduled to speak. I had few demos in my talk, and I needed to set everything up before the talk. After the tea break, I attended the session by Pratap Vardhan on “Consuming Government Data with Python and D3”. The insights from the data and the analysis by Gramener were quite interesting.

I then moved to the “dgplug” 1130 IST staircase meeting, where I met more #dgplug students, and was able to connect IRC nicknames to people’s faces. My suggestion to the students was to start to identify their interests, and to specialize on them. Around noon, I moved back to auditorium-1 to setup everything that I needed for my talk - “Pretty Printing in Python”. There is one slot which I never wanted - the one before lunch because I know that I will be the one standing between the audience and their lunch, and that is exactly what I got. Nevertheless, the slot had to be taken by someone, and I was prepared for it. I had requested the organizers for a whiteboard and markers for my talk, and they had arranged for the same. I had lot of material to cover in 45 minutes. I did rush through, but, paused at times to explain the concepts. The audience were quite interested in 3D printing, and they asked some really interesting questions at the end of the talk. The video is available in YouTube:

After a quick lunch, I headed for the lightning talks to present “Nursery Rhymes”. We are all familiar with nursery rhymes, and the idea is to teach programming languages and software development using them. The audience needed something to wake them after a heavy lunch, and they thoroughly enjoyed the singing! The video link is provided below:

I then attended the “Machine learning techniques for building a large scale production ready prediction system using Python” by Arthi Venkataraman. After a short tea break, I attended the talk by Pooja Salpekar on “Test Driven Development with Ansible”. She gave an overview of using Serverspec. The subject to test is still determined by the developer, and testing with Vagrant is not the same as testing in production deployments. For the last session of the day, I attended the “Python and Riak DB, a perfect couple for huge scale distributed computing” by Naren Arya, but, it was yet another generic talk.

In the evening, I participated in the discussion on building and running communities organized by Satyaakam Goswami on how Python communities are organizing themselves in India, and the challenges faced by them. I shared my experience with the Chennaipy meet-ups, and insights from my “i want 2 do project. tell me wat 2 do” presentation and book. A video recording of the discussion is available:

Day 3

Nicholas H. Tollervey began the day’s proceedings with his keynote. He shared his experience on Python and education. The Micro Python and BBC micro:bit projects are interesting initiatives. The audience did have questions on why they cannot use Android or any other mobile phone for Python development. But, I think they missed the point that Open Hardware initiatives are important and the openness is not the same with present day mobile devices. Nicholas also covered important points on how to organize Python Dojo meet-ups, and as a teacher, the experiences he shared were very useful.

After the tea break, I attended the session by Sumith Kulal on “SymEngine: The future fast core of computer algebra systems”. It was an introductory session on SymEngine, and he shared the roadmap and plan that they have for the software. I was then networking with people before attending “How to build microservices using ZeroMQ and WSGI” by Srinath G. S. It was yet another generic talk, and I headed for early lunch.

The previous night I prepared for another lightning talk titled “Yet Another Lightning Talk”, to once again, wake the audience after their heavy lunch. The talk went as I had planned, and the audience loved it! The video recording is available in YouTube:

A number of people approached me after the lightning talk, appreciating the creativity, and I did make a lot of new friends. The last talk of the day that I attended was on “Python load balancer; 0 to 1 million requests per second”, and yet again it turned out to be a a very generic talk, without going into specifics. I left early to catch a flight.

It will be helpful if organizers can identify talks where the speaker has contributed to the project. When you hire people for free and open source software (F/OSS) work, you would like to see what their contribution to the project is, and how they have added value to the project. I am interested to hear on what the speaker has done. I can always read the project web site, and find out more information about the project. I wish all conferences add this criteria in their talk selection process.

The workshops have been useful. The content of the talks need to be improved though. I was able to meet fellow F/OSS developers during the conference, and also made new friends. I am pleased that I was able to deliver a talk, and two lightning talks.

I would like to thank Manufacturing System Insights for sponsoring my travel, and allowing me to 3D print during office hours!

I had given a couple of talks at the Chennaipy meetups at the Institute of Mathematical Sciences, Chennai leading up to my talk at PyCon India 2015.

August 2015 meetup

The first talk was titled “Paho Python client for MQTT” presented on Saturday, August 22, 2015. The slides and notes for the presentation are given below:

MQTT Architecture

pub-sub-model.png

Source: Device to Cloud: MQTT and the power of topic notation

Topics

$SYS/broker/load/connections
$SYS/broker/load/messages/received
$SYS/broker/load/messages/sent

home/living-room/temperature
home/kitchen/humidity
home/store-room/brightness

MQTT Protocol

connect-flow.png

Source: MQTT Essentials Part 3: Client, Broker and Connection Establishment

subscribe_flow.png

Source: MQTT Essentials Part 4: MQTT Publish, Subscribe & Unsubscribe

Mosquitto MQTT Installation

$ sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa

  You are about to add the following PPA to your system:
 
  More info: https://launchpad.net/~mosquitto-dev/+archive/ubuntu/mosquitto-ppa
  Press [ENTER] to continue or ctrl-c to cancel adding it

  gpg: keyring `/tmp/tmpd67wf6/secring.gpg' created
  gpg: keyring `/tmp/tmpd67wf6/pubring.gpg' created
  gpg: requesting key 262C4500 from hkp server keyserver.ubuntu.com
  gpg: /tmp/tmpd67wf6/trustdb.gpg: trustdb created
  gpg: key 262C4500: public key "Launchpad mosquitto" imported
  gpg: Total number processed: 1
  gpg:               imported: 1  (RSA: 1)
  OK

$ apt-get update
$ apt-get install python-pip mosquitto mosquitto-clients
$ pip install paho-mqtt

Connectivity

Start mosquitto server

$ sudo /etc/init.d/mosquitto status
mosquitto stop/waiting

$ sudo /etc/init.d/mosquitto start
mosquitto start/running, process 11346

Verification

$ netstat -na | grep :1883
tcp        0      0 0.0.0.0:1883            0.0.0.0:*               LISTEN     
tcp6       0      0 :::1883                 :::*                    LISTEN 

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 14.10
Release:	14.10
Codename:	utopic

Pub/Sub

Subscribe

$ mosquitto_sub -d -t hello/world

Client mosqsub/11678-achilles sending CONNECT
Client mosqsub/11678-achilles received CONNACK
Client mosqsub/11678-achilles sending SUBSCRIBE (Mid: 1, Topic: hello/world, QoS: 0)
Client mosqsub/11678-achilles received SUBACK
Subscribed (mid: 1): 0

Publish

$ mosquitto_pub -d -t hello/world -m "Vanakkam"
Client mosqpub/11712-achilles sending CONNECT
Client mosqpub/11712-achilles received CONNACK
Client mosqpub/11712-achilles sending PUBLISH (d0, q0, r0, m1, 'hello/world', \
  ... (8 bytes))
Client mosqpub/11712-achilles sending DISCONNECT

Verification

Client mosqsub/11678-achilles received PUBLISH (d0, q0, r0, m0, 'hello/world', \
  ... (8 bytes))
Vanakkam

Paho Python connectivity

  import paho.mqtt.client as mqtt

  def on_connect(client, userdata, flags, rc):
      print("Connected with result code "+str(rc))
      client.subscribe("hello/world")

  def on_message(client, userdata, msg):
      print(msg.topic+" "+str(msg.payload))

  client = mqtt.Client()
  client.on_connect = on_connect
  client.on_message = on_message

  client.connect("localhost", 1883, 60)

  client.loop_forever()
$ python client.py
$ mosquitto_pub -d -t hello/world -m "Hello, World"

iot.eclipse.org

  import paho.mqtt.client as mqtt

  def on_connect(client, userdata, flags, rc):
      print("Connected with result code "+str(rc))
      client.subscribe("$SYS/#")

  def on_message(client, userdata, msg):
      print(msg.topic+" "+str(msg.payload))

  client = mqtt.Client()
  client.on_connect = on_connect
  client.on_message = on_message

  client.connect("iot.eclipse.org", 1883, 60)

  client.loop_forever()

Source Code

  paho-mqtt-1.1:
  -rw-rw-r-- 1 shakthi shakthi  2031 Feb  1  2015 about.html
  -rw-rw-r-- 1 shakthi shakthi  3736 Feb  1  2015 CONTRIBUTING.md
  -rw-rw-r-- 1 shakthi shakthi  1569 Feb  1  2015 edl-v10
  -rw-rw-r-- 1 shakthi shakthi 11695 Feb  1  2015 epl-v10
  drwxrwxr-x 2 shakthi shakthi  4096 Feb  1  2015 examples
  -rw-rw-r-- 1 shakthi shakthi   156 Feb  1  2015 LICENSE.txt
  -rw-rw-r-- 1 shakthi shakthi  9230 Feb  1  2015 notice.html
  -rw-rw-r-- 1 shakthi shakthi   814 Feb  1  2015 PKG-INFO
  -rw-rw-r-- 1 shakthi shakthi 30871 Feb  1  2015 README.rst
  -rw-rw-r-- 1 shakthi shakthi   972 Feb  1  2015 setup.py
  drwxrwxr-x 3 shakthi shakthi  4096 Feb  1  2015 src

Embedded

raspberry-small.jpg3g.pngrfid.png arduino.jpgradiation.pngbluetooth.png

Source: Raspberry Pi to Arduino

References

September 2015 meetup

The September 2015 talk was scheduled on Saturday, September 26, 2015 and I gave a talk on “G-code Visualization Tools”. The talk began with an introduction and history of G-codes. A Maya pyramid STL file was then used with Pronterface to generate the following G-codes:

G90         ; Set to Absolute Positioning
G21         ; Set Units to Millimeters
M103        ; Turn off extruder
M105        ; Get extruder temperature
M106        ; Fan (cooling) on
M140 S60.0  ; Set Bed temperature
M141 S30.0  ; Set Chamber temperature
M142 S0.0   ; Holding pressure
M113 S1.0   ; Set Extruder PWM
M108 S210.0 ; Set Extruder Speed
M104 S200.0 ; Set Extruder Temperature
G1 X-54.72 Y-53.352 Z0.72 F60.0 ; G1 Move, F Feedrate, mm/rev
M101        ; Turn extruder 1 on
G1 X-54.72 Y53.352 Z0.72 F240.0
G1 X-51.84 Y53.352 Z0.72 F240.0
...

I then gave a demo of “Yet Another GCode Viewer” written by Jonathan Winterflood that uses the Pyglet library to render the G-code paths. You can also zoom, rotate and scale the image after it has loaded the input data. The brim that is printed for 3D objects is clearly visible in the rendition. A screenshot is shown below:

YAGV

Blender is a free and open source software for computer graphics, 3D modelling and animations. It is written in C, C++ and Python. A Blender gcode reader plugin written by Simon Kirkby exists to visualize G-codes. This plugin generates the image from the G-codes, and converts the change in paths into arcs for a smoother image. The advantage of this plugin is that you can use all the goodies from Blender and Python. A screenshot of the pyramid using the Blender plugin is shown below:

G-code Blender plugin

The Visualization Toolkit (VTK) is yet another free and open source software for image processing and visualization. It requires its input file in a VTK file format. After using the gcode2vtk utility to convert the Maya pyramid G-code file to a VTK file format, I was able to demonstrate the G-code path simulation using ParaView - a data analysis and visualization application. An illustration of the Maya pyramid in ParaView is shown below:

G-code ParaView

ParaView is written in C, C++, Fortran and Python. Both Blender and ParaView allow you to run simulations on the G-codes.

All the visualization tools discussed are free and open source software, and you can install them on your favourite *nix system.

[Published in Open Source For You (OSFY) magazine, January 2015 edition.]

In this article we shall cover testing of Haskell programs.

HUnit is a unit testing framework available for Haskell. It is similar to JUnit, which is used for the Java programming language. You can install HUnit on Fedora using the following command:

$ sudo yum install ghc-HUnit-devel

Consider a simple example that follows:

import Test.HUnit

test1 = TestCase $ assertEqual "Test equality" 3 (2 + 1)

The TestCase is a constructor defined in the Test data type. The definition is as follows:

-- Test Definition
-- ===============

-- | The basic structure used to create an annotated tree of test cases.
data Test
    -- | A single, independent test case composed.
    = TestCase Assertion
    -- | A set of @Test@s sharing the same level in the hierarchy. 
    | TestList [Test]
    -- | A name or description for a subtree of the @Test@s.
    | TestLabel String Test

On executing the above code with GHCi, you get the following output:

$ ghci test.hs 

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( one.hs, interpreted )
Ok, modules loaded: Main.

ghci> runTestTT test1

Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package HUnit-1.2.5.2 ... linking ... done.
Cases: 1  Tried: 1  Errors: 0  Failures: 0
Counts {cases = 1, tried = 1, errors = 0, failures = 0}

You can build a test suite of tests with TestList as shown below:

import Test.HUnit

test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
                 , "Test subtraction" ~: 3 ~=? (4 - 1)
                 ]

The ‘~=?’ operation is shorthand to assert equality. Its definition is as follows:

-- | Shorthand for a test case that asserts equality (with the expected 
--   value on the left-hand side, and the actual value on the right-hand
--   side).
(~=?) :: (Eq a, Show a) => a     -- ^ The expected value 
                        -> a     -- ^ The actual value
                        -> Test

The ‘~:’ operation is shorthand to attach a label to a test. Its definition is shown below:

(~:) :: (Testable t) => String -> t -> Test
label ~: t = TestLabel label (test t)

By compiling and executing the above code with GHCi, you get the following result:

ghci> runTestTT test1

Cases: 2  Tried: 2  Errors: 0  Failures: 0
Counts {cases = 2, tried = 2, errors = 0, failures = 0}

A failure is reported when there is a mismatch between the expected result and the observed value. For example:

import Test.HUnit

test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
                 , "Test subtraction" ~: 3 ~=? (4 - 2)
                 ]

Running the above code reports the failure in the output:

ghci> runTestTT test1

### Failure in: 1:Test subtraction
expected: 3
 but got: 2
Cases: 2  Tried: 2  Errors: 0  Failures: 1
Counts {cases = 2, tried = 2, errors = 0, failures = 1}

If our test case definition is incorrect, the compiler will throw an error during compilation time itself! For example:

import Data.Char
import Test.HUnit

test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
                 , "Test case" ~: "EARTH" ~=? (map "earth")
                 ]

On compiling the above code, you get the following output:

ghci> :l error.hs

[1 of 1] Compiling Main             ( error.hs, interpreted )

error.hs:5:48:
    Couldn't match expected type `[Char]'
                with actual type `[a1] -> [b0]'
    In the return type of a call of `map'
    Probable cause: `map' is applied to too few arguments
    In the second argument of `(~=?)', namely `(map "earth")'
    In the second argument of `(~:)', namely
      `"EARTH" ~=? (map "earth")'

error.hs:5:52:
    Couldn't match expected type `a1 -> b0' with actual type `[Char]'
    In the first argument of `map', namely `"earth"'
    In the second argument of `(~=?)', namely `(map "earth")'
    In the second argument of `(~:)', namely
      `"EARTH" ~=? (map "earth")'
Failed, modules loaded: none.

The correct version of the code and its output are shown below:

import Data.Char
import Test.HUnit

test1 = TestList [ "Test addition" ~: 3 ~=? (2 + 1)
                 , "Test case" ~: "EARTH" ~=? (map toUpper "earth")
                 ]

The expected test output is as follows:

ghci> runTestTT test1

Cases: 2  Tried: 2  Errors: 0  Failures: 0
Counts {cases = 2, tried = 2, errors = 0, failures = 0}

In Haskell, one needs to check for an empty list when using the head function, or else it will throw an error. An example test is shown below:

import Test.HUnit

test1 = TestCase $ assertEqual "Head of emptylist" 1 (head [])

Executing the above code gives:

ghci> runTestTT test1

### Error:
Prelude.head: empty list
Cases: 1  Tried: 1  Errors: 1  Failures: 0
Counts {cases = 1, tried = 1, errors = 1, failures = 0}

Other than the assertEqual function, there are conditional assertion functions like assertBool, assertString and assertFailure that you can use. The assertBool function takes a string that is displayed if the assertion fails and a condition to assert. A couple of examples are shown below:

import Test.HUnit

test1 = TestCase $ assertBool "Does not happen" True

Executing the above code in GHCi, gives you the following output:

ghci> runTestTT test1

Cases: 1  Tried: 1  Errors: 0  Failures: 0
Counts {cases = 1, tried = 1, errors = 0, failures = 0}

Consider the case when there is a failure:

import Test.HUnit

test1 = TestCase $ assertBool "Failure!" (False && False)

The corresponding output is shown below:

ghci> runTestTT test1

### Failure:
Failure!
Cases: 1  Tried: 1  Errors: 0  Failures: 1
Counts {cases = 1, tried = 1, errors = 0, failures = 1}

assertFailure and assertString functions take a string as input, and return an Assertion. These are used as part of other test functions. The assertFailure function is used in the definition of the assertBool function as shown below:

-- | Asserts that the specified condition holds.
assertBool :: String    -- ^ The message that is displayed if the assertion fails
           -> Bool      -- ^ The condition
           -> Assertion
assertBool msg b = unless b (assertFailure msg)

You can also use assertString for handling a specific case. For example:

import Test.HUnit

test1 = TestCase $ assertString "Failure!"

Executing the above code in GHCi results in the following failure message:

ghci> runTestTT test1

### Failure:
Failure!
Cases: 1  Tried: 1  Errors: 0  Failures: 1
Counts {cases = 1, tried = 1, errors = 0, failures = 1}

Hspec is another testing framework for Haskell, similar to Ruby’s RSpec. You can install it on any GNU/Linux distribution using the Cabal tool:

$ cabal update && cabal install hspec hspec-contrib

A simple example is shown below:

import Test.Hspec

main :: IO ()
main = hspec $ do
  describe "Testing equality" $ do
    it "returns 3 for the sum of 2 and 1" $ do
      3 `shouldBe` (2 + 1)

Executing the above code with GHCi produces the following verbose output:

$ ghci spec.hs 

GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( spec.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Loading package array-0.4.0.1 ... linking ... done.
Loading package deepseq-1.3.0.1 ... linking ... done.
Loading package HUnit-1.2.5.2 ... linking ... done.
Loading package old-locale-1.0.0.5 ... linking ... done.
Loading package time-1.4.0.1 ... linking ... done.
Loading package random-1.0.1.1 ... linking ... done.
Loading package transformers-0.3.0.0 ... linking ... done.
Loading package stm-2.4.2 ... linking ... done.
Loading package bytestring-0.10.0.2 ... linking ... done.
Loading package unix-2.6.0.1 ... linking ... done.
Loading package containers-0.5.0.0 ... linking ... done.
Loading package pretty-1.1.1.0 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package QuickCheck-2.6 ... linking ... done.
Loading package ansi-terminal-0.6.2.1 ... linking ... done.
Loading package async-2.0.1.6 ... linking ... done.
Loading package hspec-expectations-0.6.1 ... linking ... done.
Loading package quickcheck-io-0.1.1 ... linking ... done.
Loading package setenv-0.1.1.1 ... linking ... done.
Loading package primitive-0.5.0.1 ... linking ... done.
Loading package tf-random-0.5 ... linking ... done.
Loading package hspec-core-2.0.2 ... linking ... done.
Loading package hspec-discover-2.0.2 ... linking ... done.
Loading package hspec-2.0.2 ... linking ... done.

Testing equality
  returns 3 for the sum of 2 and 1

Finished in 0.0002 seconds
1 example, 0 failures

The keywords describe and it are used to specify the tests. The context keyword can also be used as an alias for describe. A particular test can be marked ‘pending’ using the pending and pendingWith keywords. The pendingWith function takes a string message as an argument, as illustrated below:

import Test.Hspec

main = hspec $ do
  describe "Testing equality" $ do
    it "returns 3 for the sum of 2 and 1" $ do
      3 `shouldBe` (2 + 1)

  describe "Testing subtraction" $ do
    it "returns 3 when subtracting 1 from 4" $ do
      pendingWith "need to add test"

The corresponding output for the above code snippet is provided below:

ghci> main

Testing equality
  returns 3 for the sum of 2 and 1
Testing subtraction
  returns 3 when subtracting 1 from 4
     # PENDING: need to add test

Finished in 0.0006 seconds
2 examples, 0 failures, 1 pending

You can use the after and before keywords to specify setup and tear down functions before running a test. For example:

import Test.Hspec

getInt :: IO Int
getInt = do
  putStrLn "Enter number:"
  number <- readLn
  return number
 
afterPrint :: ActionWith Int
afterPrint 3 = print 3
 
main = hspec $ before getInt $ after afterPrint $ do
  describe "should be 3" $ do
    it "should successfully return 3" $ \n -> do
      n `shouldBe` 3

Executing the above code yields the following output:

ghci> main

should be 3
Enter number:
3
3
  should successfully return 3

Finished in 0.6330 seconds
1 example, 0 failures

The different options used with Hspec can be listed with the –help option, as follows:

$ runhaskell file.hs --help
Usage: spec.hs [OPTION]...

OPTIONS
                --help              display this help and exit
  -m PATTERN    --match=PATTERN     only run examples that match given PATTERN
                --color             colorize the output
                --no-color          do not colorize the output
  -f FORMATTER  --format=FORMATTER  use a custom formatter; this can be one of:
                                       specdoc
                                       progress
                                       failed-examples
                                       silent
  -o FILE       --out=FILE          write output to a file instead of STDOUT
                --depth=N           maximum depth of generated test values for
                                    SmallCheck properties
  -a N          --qc-max-success=N  maximum number of successful tests before a
                                    QuickCheck property succeeds
                --qc-max-size=N     size to use for the biggest test cases
                --qc-max-discard=N  maximum number of discarded tests per
                                    successful test before giving up
                --seed=N            used seed for QuickCheck properties
                --print-cpu-time    include used CPU time in summary
                --dry-run           pretend that everything passed; don't verify
                                    anything
                --fail-fast         abort on first failure
  -r            --rerun             rerun all examples that failed in the
                                    previously test run (only works in GHCi)

Other than the shouldBe expectation, you can also use assertions like shouldReturn, shouldSatisfy, and shouldThrow. Examples of each are given below:

import Test.Hspec
import Control.Exception (evaluate)

main = hspec $ do
  describe "shouldReturn" $ do
    it "should successfully return 3" $ do
      return 3 `shouldReturn` 3

  describe "shouldSatisfy" $ do
    it "should satisfy the condition that 10 is greater than 5" $ do
      10 `shouldSatisfy` (> 5)

  describe "shouldThrow" $ do
    it "should throw an exception when taking head of an empty list" $ do
      evaluate (1 `div` 0) `shouldThrow` anyException                                     

Executing the above code provides the following output:

ghci> main

shouldReturn
  should successfully return an Integer
shouldSatisfy
  should satisfy the condition that 10 is greater than 5
shouldThrow
  should throw an exception when taking head of an empty list

Finished in 0.0015 seconds
3 examples, 0 failures

The evaluate function can be used to check for exceptions. Its type signature is as follows:

ghci> :t evaluate
evaluate :: a -> IO a

You can also re-use HUnit tests and integrate them with Hspec. An example is shown below:

import Test.HUnit
import Test.Hspec
import Test.Hspec.Contrib.HUnit (fromHUnitTest)

test1 = TestList [ TestLabel "Test subtraction" foo ]

foo :: Test
foo = TestCase $ do
  3 @?= (4 - 1)

main = hspec $ do
  describe "Testing equality" $ do
    it "returns 3 for the sum of 2 and 1" $ do
      3 `shouldBe` (2 + 1)

  describe "Testing subtraction" $ do
    fromHUnitTest test1

Testing the code in GHCi produces the following output:

ghci> main
Loading package hspec-contrib-0.2.0 ... linking ... done.

Testing equality
  returns 3 for the sum of 2 and 1
Testing subtraction
  Test subtraction

Finished in 0.0004 seconds
2 examples, 0 failures

The Hspec tests can also be executed in parallel. For example, computing the Fibonacci for a set of numbers, and asserting their expected values can be run in parallel as shown below:

import Test.Hspec

fib :: Int -> Int
fib 0 = 0
fib 1 = 1
fib n = fib (n-1) + fib (n-2)

main = hspec $ parallel $ do
  describe "Testing Fibonacci" $ do
    it "must return 6765 for fib 20" $ do
      fib 20 `shouldBe` 6765

    it "must return 75025 for fib 25" $ do
      fib 25 `shouldBe` 75025

    it "must return 832040 for fib 30" $ do
      fib 30 `shouldBe` 832040

    it "must return 9227465 for fib 35" $ do
      fib 35 `shouldBe` 9227465

You can compile and execute the above code as shown below:

$ ghc -threaded parallel.hs 
Linking parallel ...

$ ./parallel +RTS -N -RTS

Testing Fibonacci
  must return 6765 for fib 20
  must return 75025 for fib 25
  must return 832040 for fib 30
  must return 9227465 for fib 35

Finished in 0.9338 seconds
4 examples, 0 failures

[Published in Open Source For You (OSFY) magazine, December 2014 edition.]

This article is a must read for anyone interested in getting a good insight into the input/output (IO) functionality of Haskell.

Input/output (IO) can cause side-effects and hence is implemented as a Monad. The IO Monad takes some input, does some computation and returns a value. The IO action is performed inside a main function. Consider a simple ‘Hello world’ example:

main = putStrLn "Hello, World!"

Executing the above code in GHCi produces the following output:

$ ghci hello.hs
GHCi, version 7.6.3: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
[1 of 1] Compiling Main             ( foo.hs, interpreted )
Ok, modules loaded: Main.

ghci> main
Hello, World!

The type signatures of main and putStrLn are:

main :: IO ()

putStrLn :: String -> IO ()

putStrLn takes a String as input and prints the String to output. It doesn’t return anything, and hence the return type is the empty tuple ().

The getLine function performs an IO to return a String.

ghci> :t getLine
getLine :: IO String

ghci> name <- getLine
Foo

ghci> name
"Foo"

The ‘<-’ extracts the result of the IO String action, unwraps it to obtain the String value, and ‘name’ gets the value. So, the type of ‘name’ is:

ghci> :t name
name :: String

The do syntax is useful to chain IO together. For example:

main = do
  putStrLn "Enter your name:"
  name <- getLine
  putStrLn ("Hello " ++ name)

Executing the code in GHCi gives the following results:

ghci> main
Enter your name:
Shakthi
Hello Shakthi

The putStr function is similar to the putStrLn function, except that it doesn’t emit the newline after printing the output string. Its type signature and an example are shown below:

ghci> :t putStr
putStr :: String -> IO ()

ghci> putStr "Alpha "
Alpha ghci> 

The putChar function takes a single character as input, and prints the same to the output. For example:

ghc> :t putChar
putChar :: Char -> IO ()

ghci> putChar 's'
s 

The getChar function is similar to the getLine function except that it takes a Char as input. Its type signature and usage are illustrated below:

ghci> :t getChar
getChar :: IO Char

ghci> a <- getChar
d

ghci> a
'd'

ghci> :t a
a :: Char

The print function type signature is as follows:

ghci> :t print
print :: Show a => a -> IO ()

It is a parameterized function which can take input of any type that is an instance of the Show type class, and prints that to the output. Some examples are given below:

ghci> print 1
1

ghci> print 'c'
'c'

ghci> print "Hello"
"Hello"

ghci> print True
True

The getContents function reads the input until the end-of-file (EOF) and returns a String. Its type signature is shown below:

ghci> :t getContents
getContents :: IO String

An example code is demonstrated below. It only outputs lines whose length is less than five characters:

main = do
  putStrLn "Enter text:"
  text <- getContents
  putStr . unlines . filter (\line -> length line < 5) $ lines text 

Testing the above example gives the following output:

ghci> main
Enter text:
a
a

it
it

the
the

four
four

empty

twelve

haskell

o
o

You can break out of this execution by pressing Ctrl-c at the GHCi prompt.

The openFile, hGetContents, hClose functions can be used to obtain a handle for a file, to retrieve the file contents, and to close the handle respectively. This is similar to file handling in C. Their type signatures are shown below:

ghci> :m System.IO

ghci> :t openFile
openFile :: FilePath -> IOMode -> IO Handle

ghci> :t hGetContents
hGetContents :: Handle -> IO String

ghci> :t hClose
hClose :: Handle -> IO ()

The different IOModes are ReadMode, WriteMode, AppendMode and ReadWriteMode. It is defined as follows:

-- | See 'System.IO.openFile'
data IOMode      =  ReadMode | WriteMode | AppendMode | ReadWriteMode
                    deriving (Eq, Ord, Ix, Enum, Read, Show)

An example code is illustrated below:

import System.IO

main = do
  f <- openFile "/etc/resolv.conf" ReadMode
  text <- hGetContents f
  putStr text
  hClose f

Executing the code in GHCi produces the following output:

ghci> main

# Generated by NetworkManager
nameserver 192.168.1.1

A temporary file can be created using the openTempFile function. It takes as input a directory location, and a pattern String for the filename. Its type signature is as follows:

ghci> :t openTempFile
openTempFile :: FilePath -> String -> IO (FilePath, Handle)

An example is shown below:

import System.IO
import System.Directory(removeFile)

main = do
  (f, handle) <- openTempFile "/tmp" "abc"
  putStrLn f
  removeFile f
  hClose handle

You must ensure to remove the file after using it. An example is given below:

ghci> main
/tmp/abc2731

The operations on opening a file to get a handle, getting the contents, and closing the handle can be abstracted to a higher level. The readFile and writeFile functions can be used for this purpose. Their type signatures are as follows:

ghci> :t readFile
readFile :: FilePath -> IO String

ghci> :t writeFile
writeFile :: FilePath -> String -> IO ()

The /etc/resolv.conf file is read and written to /tmp/resolv.conf in the following example:

main = do
  text <- readFile "/etc/resolv.conf"
  writeFile "/tmp/resolv.conf" text

You can also append to a file using the appendFile function:

ghci> :t appendFile
appendFile :: FilePath -> String -> IO ()

An example is shown below:

main = do
  appendFile "/tmp/log.txt" "1"
  appendFile "/tmp/log.txt" "2"
  appendFile "/tmp/log.txt" "3"

The contents of /tmp/log.txt is ‘123’.

The actual definitions of readFile, writeFile and appendFile are in the System.IO module in the Haskell base package:

readFile        :: FilePath -> IO String
readFile name   =  openFile name ReadMode >>= hGetContents

writeFile :: FilePath -> String -> IO ()
writeFile f txt = withFile f WriteMode (\ hdl -> hPutStr hdl txt)

appendFile      :: FilePath -> String -> IO ()
appendFile f txt = withFile f AppendMode (\ hdl -> hPutStr hdl txt)

The System.Environment module has useful functions to read command line arguments. The getArgs function returns an array of arguments passed to the program. The getProgName provides the name of the program being executed. Their type signatures are shown below:

ghci> :m System.Environment

ghci> :t getArgs
getArgs :: IO [String]

ghci> :t getProgName
getProgName :: IO String

Here is an example:

import System.Environment

main = do
  args <- getArgs
  program <- getProgName
  putStrLn ("Program : " ++ program)
  putStrLn "The arguments passed are: "
  mapM putStrLn args

Executing the above listed code produces the following output:

$ ghc --make args.hs 

[1 of 1] Compiling Main             ( args.hs, args.o )
Linking args ...

$ ./args 1 2 3 4 5

Program : foo
The arguments passed are: 
1
2
3
4
5

The mapM function is the map function that works for Monads. Its type signature is:

ghci> :t mapM
mapM :: Monad m => (a -> m b) -> [a] -> m [b]

The System.Directory module has functions to operate on files and directories. A few examples are shown below:

ghci> :t createDirectory
createDirectory :: FilePath -> IO ()

ghci> createDirectory "/tmp/foo"
ghci>

If you try to create a directory that already exists, it will return an exception.

ghci>  createDirectory "/tmp/bar"
*** Exception: /tmp/bar: createDirectory: already exists (File exists)

You can use the createDirectoryIfMissing function, and pass a Boolean option to indicate whether to create the directory or not. Its type signature is as follows:

ghci> :t createDirectoryIfMissing
createDirectoryIfMissing :: Bool -> FilePath -> IO ()

If True is passed and the directory does not exist, the function will create parent directories as well. If the option is False, it will throw up an error.

ghci> createDirectoryIfMissing False "/tmp/a/b/c"
*** Exception: /tmp/a/b/c: createDirectory: does not exist (No such file or directory)

ghci> createDirectoryIfMissing True "/tmp/a/b/c"
ghci>

You can remove directories using the removeDirectory or removeDirectoryRecursive functions. Their type signatures are as follows:

ghci> :t removeDirectory
removeDirectory :: FilePath -> IO ()

ghci> :t removeDirectoryRecursive
removeDirectoryRecursive :: FilePath -> IO ()

A few examples are shown below:

ghci> createDirectoryIfMissing True "/tmp/a/b/c" 
ghci>

ghci> removeDirectory "/tmp/a"
*** Exception: /tmp/a: removeDirectory: unsatisified constraints (Directory not empty)

ghci> removeDirectoryRecursive "/tmp/a"
ghci>

The existence of a file can be tested with the doesFileExist function. You can check if a directory is present using the doesDirectoryExist function. Their type signatures are:

ghci> :t doesFileExist 
doesFileExist :: FilePath -> IO Bool

ghci> :t doesDirectoryExist
doesDirectoryExist :: FilePath -> IO Bool

Some examples of using these functions are shown below:

ghci> doesDirectoryExist "/abcd"
False

ghci> doesDirectoryExist "/tmp"
True

ghci> doesFileExist "/etc/resolv.conf"
True

ghci> doesFileExist "/etc/unresolv.conf"
False

To know the current directory from where you are running the command, you can use the getCurrentDirectory function, and to know the contents in a directory you can use the getDirectoryContents function. Their type signatures are:

ghci> :t getCurrentDirectory
getCurrentDirectory :: IO FilePath

ghci> :t getDirectoryContents
getDirectoryContents :: FilePath -> IO [FilePath]

For example:

ghci> getCurrentDirectory 
"/tmp"

ghci> getDirectoryContents "/etc/init.d"
["livesys","netconsole",".","..","network","README","functions","livesys-late","influxdb"]

The copyFile, renameFile and removeFile functions are used to copy, rename and delete files. Their type signatures are shown below:

ghci> :t copyFile
copyFile :: FilePath -> FilePath -> IO ()

ghci> :t renameFile
renameFile :: FilePath -> FilePath -> IO ()

ghci> :t removeFile
removeFile :: FilePath -> IO ()

Here is a very contrived example:

import System.Directory

main = do
  copyFile "/etc/resolv.conf" "/tmp/resolv.conf"
  renameFile "/tmp/resolv.conf" "/tmp/resolv.conf.orig"
  removeFile "/tmp/resolv.conf.orig"

To obtain the file permissions, use the getPermissions function:

ghci> :t getPermissions
getPermissions :: FilePath -> IO Permissions

ghci> getPermissions "/etc/resolv.conf"
Permissions {readable = True, writable = False, executable = False, searchable = False}

It is important to separate pure and impure functions in your code, and to include the type signatures for readability. An example is shown below:

-- Pure
square :: Int -> Int
square x = x * x

-- Impure
main = do
  putStrLn "Enter number to be squared:"
  number <- readLn
  print (square number)

The readLn function is a parameterized IO action whose type signature is:

:t readLn
readLn :: Read a => IO a

Executing the code produces the following output:

ghci> main

Enter number to be squared:
5
25

[Published in Open Source For You (OSFY) magazine, November 2014 edition.]

In this article, we shall explore more type classes in Haskell. Consider the Functor type class:

class Functor f where
      fmap :: (a -> b) -> f a -> f b

It defines a function fmap, that accepts a function as an argument that takes input of type a and returns type b, and applies the function on every type ‘a’ to produce types ‘b’. The f is a type constructor. An array is an instance of the Functor class and is defined as shown below:

instance Functor [] where
	 fmap = map

The Functor type class is used for types that can be mapped over. Examples of using the Functor type class for arrays are shown below:

ghci> fmap length ["abc", "defg"]
[3,4]

ghci> :t length
length :: [a] -> Int

ghci> map length ["abc", "defg"]
[3,4]

ghci> :t map
map :: (a -> b) -> [a] -> [b]

An instance of a Functor class must satisfy two laws. Firstly, it must satisfy the identity property where running the map over an id must return the id:

fmap id = id

For example:

ghci> id ["abc"]
["abc"]

ghci> fmap id ["abc"]
["abc"]

Second, if we compose two functions and ‘fmap’ over it, then it must be the same as mapping the first function with the Functor, and then applying the second function as shown below:

fmap (f . g) = fmap f . fmap g

This can also be written for a Functor ‘F’ as follows:

fmap (f . g) F = fmap f (fmap g F)

For example:

ghci> fmap (negate . abs) [1, 2, 3, 4, 5]
[-1,-2,-3,-4,-5]

ghci> fmap negate (fmap abs [1, 2, 3, 4, 5])
[-1,-2,-3,-4,-5]

The Maybe data type can also be an instance of the Functor class:

data Maybe a = Just a | Nothing
     deriving (Eq, Ord)

instance Functor Maybe where
    fmap f (Just x) = Just (f x)
    fmap f Nothing = Nothing

For example:

ghci> fmap (+2) (Nothing)
Nothing

ghci> fmap (+2) (Just 3)
Just 5

The two laws hold good for the Maybe data type:

ghci> id Nothing
Nothing

ghci> id Just 4
Just 4

ghci> fmap (negate . abs) (Just 4)
Just (-4)

ghci> fmap negate (fmap abs (Just 4))
Just (-4)

The Applicative type class is defined to handle cases where a function is enclosed in a Functor, like ‘Just (*2)’:

class Functor f => Applicative f where
        -- | Lift a value.
        pure :: a -> f a

        -- | Sequential application.
        (<*>) :: f (a -> b) -> f a -> f b

The ‘<$>’ is defined as a synonym for ‘fmap’:

(<$>) :: Functor f => (a -> b) -> f a -> f b
f <$> a = fmap f a

The Applicative Functor must also satisfy few mathematical laws. The Maybe data type can be an instance of the Applicative class:

 instance Applicative Maybe where
    pure = Just
    (Just f) <*> (Just x) = Just (f x)
    _        <*> _        = Nothing

A few examples of Maybe for the Applicative type class are shown below:

ghci> import Control.Applicative

ghci> Just (+2) <*> Just 7
Just 9

ghci> (*) <$> Just 3 <*> Just 4
Just 12

ghci> min <$> Just 4 <*> Just 6
Just 4

ghci> max <$> Just "Hello" <*> Nothing
Nothing

ghci> max <$> Just "Hello" <*> Just "World"
Just "World"

The Applicative Functor unwraps the values before performing an operation.

For a data type to be an instance of the Monoid type class, it must satisfy two properties:

  1. Identity value
  2. Associative binary operator
a * (b * c) = (a * b) * c

These are defined in the Monoid type class:

class Monoid a where
      mempty  :: a           -- identity
      mappend :: a -> a -> a -- associative binary operation

Lists can be a Monoid. The identity operator is [] and the associative binary operator is (++). The instance definition of lists for a Monoid is given below:

instance Monoid [a] where
	 mempty  = []
	 mappend = (++)

Some examples of lists as Monoid are shown below:

ghci> import Data.Monoid

ghci> ("a" `mappend` "b") `mappend` "c"
"abc"

ghci> "a" `mappend` ("b" `mappend` "c")
"abc"

ghci> mempty `mappend` [5]
[5]

The Monad type class takes a wrapped value and a function that does some computation after unwrapping the value, and returns a wrapped result. The Monad is a container type and hence a value is wrapped in it. The bind operation (>>=) is the important function in the Monad class that performs this operation. The ‘return’ function converts the result into a wrapped value. Monads are used for impure code where there can be side effects, for example, during a system call, performing IO etc. A data type that implements the Monad class must obey the Monad Laws. The definition of the Monad class is as follows:

class Monad m where
  (>>=) :: m a -> (a -> m b) -> m b
  (>>) :: m a -> m b -> m b
  return :: a -> m a
  fail :: String -> m a

The Maybe type is an instance of a Monad and is defined as:

instance Monad Maybe where
    return x = Just x
    Nothing >>= f = Nothing
    Just x >>= f  = f x
    fail _ = Nothing

So, when ’m’ is Maybe, and ‘a’ and ‘b’ are of type Int, the bind operation becomes:

(>>=) :: Maybe Int -> (Int -> Maybe Int) -> Maybe Int

Here’s an example of how the Maybe Monad is used:

ghci> return (Just 5)
Just 5

ghci> return Nothing
Nothing

ghci> Just 5 >>= \x -> return (x + 7)
Just 12

ghci> Nothing >>= \x -> return (x + 7)
Nothing

ghci> Just 5 >>= \x -> return (x + 7) >>= \y -> return (y + 2)
Just 14

The newtype keyword is used in Haskell to define a new data type that has only one constructor and only one field inside it. The Writer data type can be defined using the record syntax as:

newtype Writer w a = Writer { runWriter :: (a, w) }

It can be an instance of a Monad as follows:

import Data.Monoid

newtype Writer w a = Writer { runWriter :: (a, w) }

instance (Monoid w) => Monad (Writer w) where
    return x = Writer (x, mempty)
    (Writer (x,v)) >>= f = let (Writer (y, v')) = f x in Writer (y, v `mappend` v')

To test the definition, you can write a double function as shown below:

double :: Int -> Writer String Int
double x = Writer (x * 2, " doubled " ++ (show x))

You can execute it using:

ghci> runWriter $ double 3
(6," doubled 3")

ghci> runWriter $ double 3 >>= double
(12," doubled 3 doubled 6")

The evaluation for the bind operation is illustrated below:

ghci> runWriter $ double 3 >>= double
(12," doubled 3 doubled 6")

ghci> runWriter $ ((double 3) >>= double)
(12," doubled 3 doubled 6")

ghci> runWriter $ ((Writer (6, "doubled 3")) >>= double)
(12," doubled 3 doubled 6")

The arguments to runWriter are matched to the bind function definition in the Writer Monad. Thus, x == 6, v == ‘doubled 3’, and f == ‘double’. The function application of ‘f x’ is ‘double 6’ which yields ‘(12, “doubled 6”)’. Thus y is 12 and v’ is ‘doubled 6’. The result is wrapped into a Writer Monad with y as 12, and the string v concatenated with v’ to give ‘doubled 3 doubled 6’. This example is useful as a logger where you want a result and log messages appended together. As you can see the output differs with input, and hence this is impure code that has side effects.

When you have data types, classes and instance definitions, you can organize them into a module that others can reuse. To enclose the definitions inside a module, prepend them with the ‘module’ keyword. The module name must begin with a capital letter followed by a list of types and functions that are exported by the module. For example:

module Control.Monad.Writer.Class (
    MonadWriter(..),
    listens,
    censor,
  ) where

...

You can import a module in your code or at the GHCi prompt, using the following command:

import Control.Monad.Writer

If you want to use only selected functions, you can selectively import them using:

import Control.Monad.Writer(listens)

If you want to import everything except a particular function, you can hide it while importing, as follows:

import Control.Monad.Writer hiding (censor)

If two modules have the same function names, you can explicitly use the fully qualified name, as shown below:

import qualified Control.Monad.Writer 

You can then explicitly use the ‘listens’ functions in the module using Control.Monad.Writer.listens. You can also create an alias using the ‘as’ keyword:

import qualified Control.Monad.Writer as W

You can then invoke the ‘listens’ function using W.listens.

Let us take an example of the iso8601-time 0.1.2 Haskell package. The module definition is given below:

module Data.Time.ISO8601
  ( formatISO8601
  , formatISO8601Millis
  , formatISO8601Micros
  , formatISO8601Nanos
  , formatISO8601Picos
  , formatISO8601Javascript
  , parseISO8601
  ) where

It then imports few other modules:

import Data.Time.Clock (UTCTime)
import Data.Time.Format (formatTime, parseTime)
import System.Locale (defaultTimeLocale)
import Control.Applicative ((<|>))

This is followed by the definition of functions. Some of them are shown below:

-- | Formats a time in ISO 8601, with up to 12 second decimals.
--
-- This is the `formatTime` format @%FT%T%Q@ == @%%Y-%m-%dT%%H:%M:%S%Q@.
formatISO8601 :: UTCTime -> String
formatISO8601 t = formatTime defaultTimeLocale "%FT%T%QZ" t

-- | Pads an ISO 8601 date with trailing zeros, but lacking the trailing Z.
--
-- This is needed because `formatTime` with "%Q" does not create trailing zeros.

formatPadded :: UTCTime -> String
formatPadded t
  | length str == 19 = str ++ ".000000000000"
  | otherwise        = str ++ "000000000000"
  where
    str = formatTime defaultTimeLocale "%FT%T%Q" t

-- | Formats a time in ISO 8601 with up to millisecond precision and trailing zeros.
-- The format is precisely:
--
-- >YYYY-MM-DDTHH:mm:ss.sssZ
formatISO8601Millis :: UTCTime -> String
formatISO8601Millis t = take 23 (formatPadded t) ++ "Z"
...

The availability of free and open source software allows you to learn a lot from reading the source code, and it is a very essential practice if you want to improve your programming skills.

« OLDER POSTS