Something to say

Nothing to say

I sing when I’m at home. In the shower. While I cook dinner. Always badly, but I sing.

My kids, even though they’ve grown up with my constant singing, know it’s terrible and often tell me to stop. When I run the bath — which I do every night for three minutes of solitude after the rush of cooking for, and struggling to feed, my treasured monsters — I sing. There’s no one there to yell ‘stop’ or throw hands over their ears. The acoustics aren’t quite as good as the shower, but I can forget how bad I sound with the bath filling and sing in the style of people in their twenties when I was a pre-teen.

For the last three weeks I’ve been singing ‘Nothing to Say’. I’m not sure how this earworm worked its way into my repertoire in the autumn of 2017, but it was kinda funny, the way it fell just short of being ironic, that in this year of being paid to write whatever I wanted, I kept telling the bathtub I had nothing to say. I thought about writing about it. About how this song, through a quirk of musical availability in the pre-internet world, had started something.

But I didn’t write it after the first week, or the second. And then this morning I learnt Chris Cornell had killed himself in his hotel room in Detroit. Had ended his life mere hours after a Soundgarden show.

And I thought: ‘Shit, fuck, Christ.’

After a little while, I also thought, ‘Shit, I can’t write that thing anymore.’

Because I never wrote about Bowie or Scott Weiland when they died. 

Because there’s something so fucked about colonising the death of someone you’ve admired from a distance with your personal reflections. As if, by sticking your flag into the corpse, you can personalise your impersonal affections, requite your unrequited love.

But then I thought if it was just an issue of timing, of not wanting to be one of the crowd, perhaps I should still write something. I needn’t post it. At least not straight away.

I made a mental list of the musicians who meant so much to me that I’d push everything else aside and write about them. I could list three: Dave Wyndorf, Gord Downie, Chris Cornell.

So fuck it. I’ve caved. I’m basic. I’m upset. I’m writing this.

[And now, after finishing it, I’ve scrolled through my Facebook feed and no one is talking about Chris Cornell anymore and I’m thinking fuck this shitty, attention-deficient culture and I’m gonna post this now and then go offline before I talk myself out of it.]

The Encarta Years

I first heard Soundgarden on Encarta 95. The CD-ROM encyclopedia came with my family’s first PC and it would be another couple of years before we had the internet, so Encarta was basically it in the way of authoritative knowledge in my house besides maybe my parents (iffy propositions) and the newspaper.

Where Encarta beat my folks, the newspaper and even the leatherbound volumes of Encyclopedia Britannica at my Nanas, was that it had audio and video clips. 

For some reason, Encarta had a clip of ‘Nothing to Say’, in which Chris Cornell screamed tunefully (or sung screamfully) about his own in articulateness. I was twelve and shy and catching up on grunge just as its first, heady phase was ending. But Encarta didn’t have a clip from Nirvana or Pearl Jam or Alice in Chains (or Zeppelin or Sabbath or The Beatles). I think it may have had Elvis Costello’s ‘Pump it up’ (anyone?), but the only one I can remember vividly —because I returned to it whenever I was done “researching” whatever had prompted me to slide that CD into the tray — was ‘Nothing to Say’.

Soon enough I’d be sucked into Superunknown (I couldn’t decide if my favourite song was ‘The Day I Tried to Live’ or ‘Fell on Black Days’ – the answer depended on how depressed I was / how much I privileged Ben Sheppard’s bass over other elements of the band), then back though Badmotorfinger and forward through Down on the Upside.

For a long time I thought Soundgarden had two singers, one who sung and another who screamed. Later I learnt that Cornell had thought the same thing about Led Zeppelin. So there’s something to be said for growing up without the internet.

Euphoria Mourning

When my father died in 1999, I was given two jobs. Choose the music and write the eulogy. (Also: eat something!)

My father had been suffering from depression. He’d only just been diagnosed, though the doctor said he would have been suffering from it for many years. That the crisis of the past few weeks were part of something much bigger. A calmer, better-fitting black dog, but one more difficult to dislodge. He was prescribed drugs, or drugs were the next step, but then he died without ever popping a single Prozac.

In my eulogy six days later I quoted from ‘Seasons’, a solo effort from Cornell that appeared on the Singles soundtrack:

I wanna fly above the storm

But you can’t grow feathers in the rain

What I wanted to be saying, what I thought I was saying, was that I understood. That I forgave my father.

(Forgiveness —  proper, complete forgiveness — it turns out, took longer to manufacture, but now I look back at those words, and the sentiment with which I’d used them, and say, ‘Yeah, totally.’)

I included two other Cornell songs on the cassette I gave the sound guy at the church: ‘Say Hello 2 Heaven’ from Temple of the Dog (the funeral coincided with the week I believed in heaven) and ‘Sunshower’, a more recent solo Cornell offering which, I think, played after Joe Cocker’s ‘With A Little Help from My Friends’, my dad’s favourite song and the one to which we carried him out to the hearse.

Cornell’s music wasn’t my father’s (he liked Soundgarden well enough, though seemed more interested in Radiohead), but he was dead and I figured people would cut me some slack if I inserted a bit of myself into proceedings.

Later that year, Cornell released his first solo album, Euphoria Morning. I remembering listening to it in the car a lot as we drove around the country, my mum, my brother and I, staying with relatives who offered escape and distraction (for me and my brother) and a willing ear (for my mother).

Originally, Cornell was going to call the album Euphoria Mourning, but the ‘u’ was omitted in a typo at some point in production and the new title stuck. The sense of loss, though, was apparent to me. ‘Wave Goodbye’, in which Cornell does his best Jeff Buckley to farewell, uh, Jeff Buckley. ‘Preaching the End of the World’. ‘When I’m Down’. ‘Disappearing One’. Cornell had made an almost tender album for me to wallow in my grief (and the general teenage downer that I’d been on before my dad’s death), and it was wonderful.

Save yourself

In 2002 I was at university, living in a different town, living a different life. I was less imprisoned by my shyness, less hobbled by grief and my arrogance, which had bordered on misanthropy, had been chiseled away in the process of living this life.

Then I got really sick. I was coming back from University Games in Hamilton, in a van full of other students, and had to throw up in Taihape. By the time we made it back to Wellington I was a shivering wreck. My girlfriend took me to A&E and they thought it might have been meningitis. I got a spinal tap. I remember the pain. Trying not to vomit while they drained my spinal fluid. The fear.

After a shitty night in hospital they told me it wasn’t meningitis. A severe viral infection was as far as the diagnosis went before I was dismissed and left to lie in my bed for a week wracked by headaches, vomiting and diarrhea.

At some point in this hellish week I heard ‘Cochise’, the first single from Cornell’s collaboration with members of Rage Against the Machine. This was still before the internet was ubiquitous. My flatmate had it on the PC in his room, but I didn’t have my own device (I wrote all my essays in computer labs!). So when ‘Cochise’ came on the radio, it was the first I’d heard of Audioslave. And it blew me away, in spite of my physical weakness and torpor. I listened to the same station for hours until they played the song again so that I could record it onto a cassette, something I hadn’t done since leaving home.

After a skittery intro that called to mind rotor blades and scorched earth, Cornell comes in at full volume (I love the conversational ‘Well’, but the way):

Well, I've been watching
While you've been coughing
I've been drinking life
While you've been nauseous
And so I drink to health
While you kill yourself
And I've got just one thing
That I can offer
Go on and save yourself
And take it out on me

Two minutes before, three years had felt like a lifetime of distance between me and the kid I thought I’d left in Palmerston North. But here was Chris Cornell talking to me once more, thrillingly locked in his screaming register (save for a bridge that ends in the screamiest of screams), backed by a heavier, funkier band.

Had I heard about the collaboration with Morello, Commerford and Wilk before I heard the song, I’d have shaken my head and said, ‘Trainwreck’. But it felt as if Cornell had managed to bend Rage to achieve his own means. Which is pretty much what Cornell did for three records with Audioslave.

What would Zack de la Rocha do with a song like ‘Like a Stone’?

I was lost in the pages
Of a book full of death
Reading how we'll die alone
And if we're good, we'll lay to rest
Anywhere we want to go

Even the titles ring out with Cornell’s fixations: ‘Last remaining light’, ‘Drown me slowly’, ‘Heaven’s Dead’, ‘The Curse’, ‘Until we fall’, ‘Broken City’, ‘Nothing Left to Say But Goodbye’…

Far away from here

When Cornell was young, his depression had such a grip upon him that he could not leave the house.

But then he did.

And he kept leaving the house, kept producing music about depression and crucifixion, kept penning goodbyes that weren’t quite final until now.

Chris Cornell had plenty to say and he said it. I’m sad that he’s not with us anymore, that maybe he felt like no one was listening, that he couldn’t grow feathers amid this last storm.

I mean, fuck.

Seriously, fuck.

But then I listen to a song like ‘Boot Camp’, which is off Soundgarden’s last record before breaking up in 1997. It’s less than three minutes long, without sharing any traits you might associate with a sub-three-minute pop song. It starts with a two-phase, 1:14 long, intro, followed by a long, meandering verse and a soaring refrain (‘Far away, I’m far away from here’), and it suddenly wraps up.

It’s a track that could have resolved itself in seven minutes, become more of a song, but also, I suspect, harder to love.

Coda

Chris Cornell means more to me than a tiny tingle of cusper nostalgia. He’s more than the soundtrack of a morose thirteen-year-old or a grieving-sixteen-year old or an incapacitated nineteen-year-old. Because he’s all of these things. And none of them. However much I might think that I could conjure him up with my words, that there’s something redemptive in them, I can’t / there isn’t.

All that’s left is the familiar routine of grief-binging on an artist’s back catalogue and dipping in and out of the reports that will emerge over the coming days (the latest means I’m not sure I can listen to ‘Pretty Noose’ any time soon).

And this, of course, this bit, where I dutifully provide all the links to places you can get help, and then I personalise it in the hope that I can extend a life and expand, by whatever margin, the joy and music and sorrow and beauty in the world.

But seriously. Find that one person you can talk to. Don’t be afraid to ask. And don’t be afraid to be sad, to listen to sad songs or sing them to the bath as it fills.

If you’re in New Zealand, here are three places to start:

• www.depression.org.nz, phone 0800 111 757
• Lifeline, phone 0800 543 354
• Mental Health Foundation

But don’t forget the music…

-->

Burst around truth: an excursion in recurrent neural network poetry

The web interface for my (I mean Lech's) poetry machine

This post is scheduled to go live during the Dunedin Writer’s Festival event on Found Poetry. The event’s description reads:

Like hunter-gatherers armed with pens, paper and acute observation skills, six Dunedin writers have composed poetry collected from the fabric of the city. Come along and fill up on a diet of found poetry foraged for you by some of our finest.

False starts

I’ve been living in Dunedin just shy of four months, which is an awkward length of time. It’s too long to be able to feel like a tourist and blithely, touristically, make poetry without feeling every misstep. But it's too short to know exactly what I want to say, even if I’m to say it with other people’s words.

Point and case, my first misstep when I knew I had to find some poetry. The idea came to me when I first visited Forsyth Barr Stadium (for the Warriors v Bulldogs, alas, rather than a Highlanders game – spot the out of towner!). I looked at the names of the stands. There was the Speights Stand (of course), the Mitre 10 Mega Stand (fitting as the huge orange shed is visible from pretty much every lookout), the Otago Daily Times stand, which is only there when it’s needed (eg for All Blacks games, super rugby finals, not NRL games), which seemed to say something about traditional media and local media. Very quickly I thought of a range of other temporary, interstitial stands that could be inserted in the seating plan of Dunedin’s stadium to give a more meaningful map of what it means to live in Dunedin in 2017. Emersons needed a stand beside Speights, for starters. And what were the local businesses displaced by, or put out of business by, Mitre10 Mega?

But the more I looked at my seating plan, the more I realised, I didn’t know enough.

That, and making a seating plan poetic seemed to pull the thing in less meaningful, but funnier directions (The Lemonade Stand stand, The Stand By Your Man stand).

Verdict: abandon!

Moving on

My next and more enduring project was inspired by the screens in Toitu (Otago Settlers Museum) that played Dunedin Sound music videos. I sat there for ages, suddenly recognising streets and beaches of temporary hometown.

The music was something I could identify with, even before I’d moved down here. In my first week as the Burns Fellow I sat in Poppa’s Pizza and stared in awe at the orientation posters from the late eighties and early nineties: The Chills, Straitjacket Fits, The Bats, The 3Ds... It was one of those I just wasn't meant for these times times.

Back in 2008, while questing for a million words, I generated a lot of poetry with the help of Google Translate. I'd take song lyrics from bands like The Tragically Hip, Procul Harum and Monster Magnet, translate into (for example) Russian, then Indonesian, then Chinese (Simplified), then English, then Italian, then Latin, then English, then Welsh, then Hindi, then English. Each time the text slipped back into English, it’d scavenge any interesting collision of words and keep going until I’d built up a stockpile of phrases to then construct something new.

If you’re interested, here are some examples of finished poems that are available online:

• Himalyan White (started life as ‘Nights in White Satin’ and ‘Whiter Shade of Pale’)
• Glen Coe (started life as ‘Whipping Post’ by The Allman Brothers Band)
• In Time (started life as ‘Limo Wreck’ by Soundgarden)

(I played around with other things too. A series of warped Shakespearean soliloquies. Translating poems from languages I didn’t know without any help — analog or digital (‘The Bumblebee’ came from my ignorant ‘translation’ from the Turkish of Mustafa Ziyalan’s poem ‘Ad Bulamadiğim’).

This is all to say I have an interest in alternative means of text generation, the place of the writer in the process, and the relationship between song lyrics as text and poetry — so it was not a very big leap to make to decide to take lyrics from Dunedin Sound bands and generate something new from this corpus.

The final piece of the puzzle is my interest in Recurrent Neural Networks (RNN). I guess I have a soft spot for AI that can’t quite pass the Turing Test. I can listen to ChatBots talking to each other for hours. And when I read a list of recipe titles generated by an RNN that had been trained on cookbooks, I seriously lost my shit.

Match-making

So, I wanted to learn more about RNNs, and after a bit of reading up I knew I couldn’t do it alone. One of the great things about being part of a university, however temporary, is the possibility of collaboration just seems that much more achievable. I reached out to Anthony Robins in the University of Otago’s Computer Science Faculty, because his name appeared most often with respect to neural networks. We met up for coffee and he seemed excited by my project but was in the throes of editing a book and didn’t have time to help. He sent an email to five of his colleagues and after a few nibbles I met up for another coffee with Lech Szymanski.

Lech had developed a basic RNN for teaching purposes last year, fed it Pride and Prejudice and it was able to spit out Austeny prose with the right prompts. Lech offered to set up a web interface on one of the university’s servers so I could access this RNN, upload my own corpus, select how to train it (more on this in a second), then play around with the outputs.

And in a matter of days I had the keys (read: password) to this amazing text generator. In terms of what can be achieved in language generation, it’s still highly basic. But it has proved an amazing sandpit in which to cut my teeth (and to generate some poetry for tonight’s event!).

Method statement

Step 1:  Collect the text (corpus) you’re going to feed into the RNN.

For me this involved about 10 hours of scouring the internet for all available song lyrics by Dunedin Sound bands up to about 1995. This was quite hit or miss. I got quite a lot of Chills, Bats, Verlaines, Clean and Straightjacket Fits. Others (eg The Gordons, Look Blue Go Purple) were much harder to come by, a reflection of the fact these lyrics sites are heavily weighted towards contemporary puff and hosted overseas. All up, I got about 300 songs (45,000 words). I’d hoped to have about double that, and maybe someone will read this and help me out. But I didn’t have time to transcribe lyrics from some of the great bands that were unfairly under-represented in my corpus, and thems the breaks.

A note about the size: the RNN basically learns to speak English from the text you upload and train it on. If a word isn’t in the corpus, it won’t know it. If a word is only in there once, the model will only know one what that word can occur. There’s an element of randomisation in word selection, but to get unique but genuine sounding outputs requires a huge and varied corpus and a really well calibrated model. It’s fair to say my first attempts were going to be sitting at the more cursory end of the spectrum.

Step 2: Upload the .txt file containing these lyrics to the RNN.

Step 3: Specify the training parameters for the new model:

• Number of epochs – how many times the network goes over the entire training data during its training. The more epochs, the better at speaking this particular brand of English it’s going to be.
• Number of hidden neurons – the size of the network. More neurons, more capacity to store patterns, but the longer it’ll take to train.
• Predict from a sequence of X words – the network takes a fixed number of words in order to predict the next one.

For my first model, I went with the default (500 epochs, 128 hidden neurons, sequence of 5 words)

Step 4: Start training

And…

Waiting for a command...

Received command to train...

Error! Couldn't find dunedin...

Waiting for a command...

Received command to train...

Error! Couldn't find dunedin...

Waiting for a command...

Received command to train...

Error! Couldn't find dunedin...

Okay, so there were a few hiccups to start with (no spaces in .txt filenames, please).

When we got it going, the first model took around 12 hours to train, which meant I had a new toy to play with when I got into the office the next morning.

Generating text

Once you’ve trained your model, you need to give it some starting text that it will then follow on from, the number of word to generate, and the number of top most likely words to choose at random from.

Version 1

This first model, with 500 epochs of training and a fairly grunty 128 neurons, was so well schooled in the 300-ish song lyrics, that if you set the number of top most likely words to choose from at random around 5 or lower, it would very quickly start spitting out verbatim Verlaines lyrics or just saying soft bomb soft bomb soft bomb (Thanks, The Clean!) until it had dutifully fulfilled the order of words it needed to generate.

By turning up the randomness, it got a bit more interesting.

Very early on it became clear this exercise, while it might not produce the greatest poetry, it’s a really good way for humans (or me, at least) to synthesise a decent amount of text and look for patterns.

Because I was hoping to create a ‘found’ poem or two that reflected back on Dunedin, I was on the lookout for items of local colour in the generated text. But unlike the music videos, and that guitar jangle, there wasn’t a lot of quintessential Dunedin nounage in the corpus. ‘Dunedin’ never once came up. No suburbs (I don’t think). No ‘rugby’ or ‘Speights’ or ‘Mitre 10’. Which is telling, perhaps, in that this was counter-culture, even for Dunedin. The music videos on George St or St Clair beach weren’t so much parochial proclamations as the result of fiscal imperatives.

As I continued to tinker, I found the most glee in using Dunedin-themed seed words (eg ‘Welcome to Dunedin’, ‘Take Your Place In the World’), as a kind of antagonistic gesture to drawn these fiercely geo-generic lyrics into the service of overt local commentary.

But first I needed to tinker with the model.

Version 2

I got the RNN to retrain on the same corpus but only for 200 epochs, using 32 neurons and 5 word predictive sequence. The hope was that this would provide a model that would be prone to more random combinations of words, without producing entire gibberish.

And it did seem to work better, for my purposes at least.

In amongst the noise…

There's wonder whetherÊ, sweet

(Where bark enough

But(Why?,It’s hipper on once awake

…there were moments of almost poetry:

I'm burst around truth

 Tremble cos it's time for love

 We want a pain that grows

 But blind away our worries

 Nah surprise, mindless idiot

Poetry? Or a direct address to the mindless idiot fiddling with the knobs?

To generate proto-poems, I’d feed the model the same phrase three or four times, tell it to generate 40 or 50 words at different degrees of randomness, and tape them together as verses.

I’d then cut and paste the verbatim output and do a light edit, getting rid of the nonsense characters, adding spaces to awkwardly conjoined words, and fixing tenses.

Then I’d cut and paste that text and do a heavier edit, removing errant words or moving them within a line, changing pronouns to maintain the unity of the verse.

If it was going really well, I might do an even heavier edit, which would leave just the cream of the model’s poetic output.

Anyone who spends any time tinkering with methods of text generation for creative means is basically willing themselves into a state of extreme credulity. You have to take what you’re given and assign it meaning when the machine had no such intention. For me, there were two impulses, at times complimentary, at times competing. The text from the model was:

1. The disembodied voice of Dunedin itself; and/or
2. Autobiographical (i.e. the words Craig really wants to say, but didn’t know it until he pushed “Generate”

An example of a ‘poem’ using this second model with 3 rounds of editing from me:

Take your place in the world 

Take your place in the world

 Ride on the air, on sorrow, 

through the world and all you’re

 Keeping back

 You think I could pass the living touch to you?

 Done.

A plundering drive inside the water

 Look at this head,

it's really my cactus upon me

Take your place in the world, cross the blue

 And accept a light

tomorrow they came from today

take me up, love that's more than money,

just go, baby, without my sense.

Version 3

Ideally, I wouldn’t have to resort to anything but a light edit and let the model talk for itself. The biggest drawback seemed to be the limited corpus. If you could only learn to speak from 300 song lyrics, you’d struggle to be articulate no matter how many different ways you studied them.

But I’d exhausted the online well and wasn’t about to transcribe lyrics for 30 hours just to double the size of my corpus. So, in the spirit of experimentation, I added the Beatles songbook (30,000 words) to my Dunedin Sound lyrics. I chose the Beatles because they were a known quantity, the lyrics were readily available and have relatively little local flavor, and adding some more Lennon and McCartney songwriting DNA into the model could only be a good thing if the aim was to create more recognisable song lyric text.

And it worked pretty much as I’d hoped. There were hardly ever words that were obviously The Beatles' (begone Pam and Maxwell) and the dark and brooding nature of the previous models was still retained.

So I started feeding the model seeds of Dunedin and in return it gave me the raw materials for Dunedin-y poems. I still needed to edit, and the more I edited the better they became as poems (especially ones with the intent of reading aloud) — but of course I’d think that about my own editing. 

I don’t want to do much more analysis of the poems, they should speak for themselves, but I will add that, knowing what you now know about their genesis, it opens up a range of reading approachs that wouldn't normally exist, like: 

1. Looking for identifiable words or phrases from Dunedin Sound or (Beatles) songs
2. Looking for ‘the joins’ – where I might have wielded the heaviest hand in the editing for structural reasons
3. Reading my palms – the search for autobiographical elements despite my efforts to create a distance between me and the text
4. Dobbing Dunedin in – the search for the truth of Dunedin from the (mangled) mouths of its jean-clad prophets

So to close, I’ll leave you with two poems (I’m still not sure which I’ll read tonight – I’m the 6^th of 7 poets, so I’ll pick which fits best on the fly), both generated using the third model (Dunedin Sound + Beatles lyrics, 300 epochs, 32 neurons, 10 word sequence). The titles are the seed text I used for each generative act.

Dunedin 

Dunedin,

you call you with your old trees

that time cannot bite: 

Welcome to Think Small.

Dunedin the one, it's just, 

 How was I doing? Was I right?

 You’re inside me and you've got a summer!

  Rivers will rise,

They'll question you:

North Dunedin, I spend my life leaving,

trapped, refrain.

Another Lady hard from any year, any horizon.

 The artist using her

 kaleidoscope 

knows to touch the heads of my bedroom 

 for silence.

 Yeah, she doesn't.

Dunedin, 

 If I can’t get a better wave of submarine aches,

 Sail and sunset and back Monday, 

that distant other lover’s boat slips in two:

 you we I me

 I can guess what they're about to say,

It's:

        Dunedin with the night darkness,

should overtime bring you round

 all the west commences brown

easy-pleaser sling stood still

There's the little madman’s papers-Yes

 He put on your love song,

 Pyromaniac,

 Take in darkness what we seem,

a bad danger.

Dunedin with a little hand

 everybody’s ticked.

She was bringing us apart.

Dunedin, don't prove ugly!

Welcome to Dunedin 

Welcome to Dunedin at dark.

 You don't force dreams. 

People see only the thread that ever dreamed

 And, for you,

 a star.

Welcome to Dunedin. 

Ain't they run out of naked sorrow gloom?

 Now you'll stuff shaking bright eyes where 

the wind he reaps soft passes 

of a blue town, and

 Baby, you're dead.

 What? Please. 

Now gone in confuséd linger.

Welcome to Dunedin!

 [Chorus: dance to question the mind]

Welcome to Dunedin, 

The impairing ocean is now blue. 

Say you would never never do the lie by land.

 The message turned the pot satin.

 A seized day hauled off. 

We belong that tell the prayer.

Welcome to Dunedin,

Old hallows joyride and no-one 

gazing beside me come snow. 

Trust that I'll get beauty,

That any chaos will.

-->

Roll credits

Thanks very much to Lech Syzmanski and Anthony Robins. Thanks also to Clare Mabey & the team at Dunedin Writers Festival for putting me on the Found Poetry session and starting me off on this path. I have a lot of links to other interesting stuff people are doing with neural networks, but I'll save that for another time.