This weekend I bought a copy of The Language of Change by Paul Watlawick. I'm only a little way into it but I think it's a book about learning to communicate in the language of the right hemisphere of your brain. This is a subject that, for a number of reasons, has become interesting and relevant to me lately.
So it was with some interest that I read John Wiseman's post about the Regressive Imagery Dictionary:
The Regressive Imagery Dictionary (RID) is a coding scheme for text analysis that is designed to measure 'primordial' and conceptual content. Primordial thought is the kind of free-form, associative thinking involved in fantasy and dreams. Like Freud's id, I guess. Conceptual (or secondary) thought is logical, reality-based and focused on problem solving.
RID contains about 3,000 words grouped into categories that are themselves classified as primary, secondary, and emotional. A piece of text is classified by what percentage of its words fall into each category.
John posted some python source code he wrote. I decided I wanted to play with this and, being as I don't care for Python very much, wrote a Ruby version. It's not quite a port of rid.py although I clearly looked at how rid.py worked as I hacked together the Regressive IMagery Analyzer (RIMA) library. I also compared his numbers with mine using the same sample text. My analyzer seems comes out with the same results to a few decimal points.
If you want to play with it you can download it from RubyForge or install it as a gem:
sudo gem install RIMA
Here is an example of using it:
require 'rima'
analyzer = if File.exist?( File.expand_path( "~/.rima" ) )
RIMA::Analyzer.restore( "~/.rima" )
else
RIMA::Analyzer.new(
File.join( RIMA_PATH, 'rid', 'categories' ),
File.join( RIMA_PATH, 'rid', 'exceptions' )
).store( "~/.rima" )
end
results = analyzer.analyze( STDIN.read )
puts "Total words = #{results[:word_total]}"
puts "Counted words = #{results[:word_count]}"
puts ""
puts "Primary: %0.3f" % results[:classes][:primary]
puts "Secondary: %0.3f" % results[:classes][:secondary]
puts "Emotions: %0.3f" % results[:classes][:emotions]
puts ""
puts results[:sorted_scores].map { |r| "#{r[0].path.ljust(48)}=\t#{r[1]}" }.join( "\n" )
The code indexes the categories and exceptions from the RID dictionary and stores a marshalled copy in ~/.rima. I implemented a very simple search tree structure. I thought of using the Trie gem but couldn't see how to make it handle the wildcards in the RID data so I hacked something together myself.
Anyhoo, when fed Bush's speech (from the second debate with Al Gore back in 2000 the result was:
Alia:~ matt$ rima < Projects/ruby/rima/data/bush.txt
Total words = 7982
Counted words = 1401
Primary: 0.230
Secondary: 0.672
Emotions: 0.098
SECONDARY/ABSTRACT_THOUGHT = 281
SECONDARY/INSTRU_BEHAVIOR = 197
SECONDARY/SOCIAL_BEHAVIOR = 181
PRIMARY/REGR_KNOL/CONCRETENESS = 170
SECONDARY/MORAL_IMPERATIVE = 107
SECONDARY/TEMPORAL_REPERE = 87
SECONDARY/RESTRAINT = 60
EMOTIONS/AFFECTION = 57
EMOTIONS/AGGRESSION = 51
SECONDARY/ORDER = 29
PRIMARY/SENSATION/VISION = 24
PRIMARY/DEFENSIVE_SYMBOL/PASSIVITY = 23
PRIMARY/SENSATION/COLD = 20
PRIMARY/ICARIAN_IM/WATER = 13
PRIMARY/REGR_KNOL/NARCISSISM = 10
PRIMARY/SENSATION/HARD = 10
EMOTIONS/GLORY = 10
EMOTIONS/POSITIVE_AFFECT = 9
PRIMARY/ICARIAN_IM/HEIGHT = 8
EMOTIONS/ANXIETY = 7
PRIMARY/ICARIAN_IM/FIRE = 6
PRIMARY/ICARIAN_IM/DESCENT = 5
PRIMARY/SENSATION/SOUND = 4
PRIMARY/NEED/SEX = 4
PRIMARY/REGR_KNOL/BRINK-PASSAGE = 4
PRIMARY/ICARIAN_IM/DEPTH = 3
PRIMARY/DEFENSIVE_SYMBOL/VOYAGE = 3
PRIMARY/ICARIAN_IM/ASCEND = 3
PRIMARY/SENSATION/GEN_SENSATION = 3
PRIMARY/DEFENSIVE_SYMBOL/DIFFUSION = 2
EMOTIONS/SADNESS = 2
PRIMARY/NEED/ORALITY = 2
PRIMARY/DEFENSIVE_SYMBOL/RANDOM MOVEMENT = 2
PRIMARY/DEFENSIVE_SYMBOL/CHAOS = 1
PRIMARY/REGR_KNOL/COUNSCIOUS = 1
EMOTIONS/EXPRESSIVE_BEH = 1
PRIMARY/REGR_KNOL/UNKNOW = 1
Which is pretty close to the numbers Wiseman's code generates (the relative percentages of primary/secondary/emotions are I think more or less identical). The utility to generate this analysis comes with the RIMA library and is automatically installed as a binary by the gem, you can run it using:
rima < input_text
I'm going to be turning RIMA loose on Curiouser & Curiouser at some point, that might be interesting, and there are some other things I have in mind for it. But it could be fun to just noodle around with it. Please let me know if you do anything interesting with this code.
As an aside there here is an interesting exchange involving Dr. Watzlawick related by Robert Anton Wilson (in his excellent book Quantum Psychology):
Dr. Watzlawick, incidently, got his first inkling of this psychotomimetic function of semantic noise when arriving at a mental hospital as a new staff member. He reported to the office of the Chief Psychiatrist, where he found a woman sitting at the desk in the outer office. Dr. Watzlawick made the assumption he had found the boss's secretary.
"I'm Watzlawick," he said, assuming the "secretary" would know he had an appointment.
"I didn't say you were," she replied.
A bit taken aback, Dr. Watzlawick exclaimed, "But I am."
"Then why did you deny it?" she asked.
At this point, in Dr. Watzlawick's view of the situation, the woman no longer seemed a secretary. He now classified her as a schizophrenic patient who had somehow wandered into the staff offices. Naturally, he became very careful in "dealing with" her.
His revised assumption seems logical, does it not? Only poets and schizophrenics communicate in language that defies rational analysis, and poets do not normally do so in ordinary conversation, or with the above degree of opacity. They also do it was a certain elegance, lacking in this case, and usually with some kind of rhythmn and sonority.
However, from the woman's point of view, Dr. Watzlawick himself had appeared as a schizophrenic patient. You see, due to noise, she had heard a different conversation.
A strange man had approached and said, "I'm not Slavic." Many paranoids begin a conversation with such assertions, vitally important to them, but sounding a bit strange to the rest of us.
"I didn't say you were," she replied trying to soothe him.
"But I am," he shot back, thereby graduating from "paranoid" to "paranoid schizophrenic" in her judgement.
"Then why did you deny it?" She asked reasonably and became very careful in "dealing with" him.
Anybody who had experience dealing with schizophrenics will recognize how both parties in this conversation felt. Dealing with poets never has quite this much hassle.
I think R.A.W. maybe never had to deal with many poets.
My FlickrPro account is coming up for renewal in early June. As someone who is only tangentially a photographer (and mainly of my cats in any case) it's not vital to me and, frankly, I've seen nothing good happen to Flickr since the Yahoo acquisition. The free account seems pretty much crippled to me so I think I will delete the account and rehost anything I still think is worth sharing somewhere else.
Whatever my qualms about Google search, I find Google maps quite invaluable. I'm often looking up places people talk about to see where they are in relation to me or some other point of interest.
Where GMaps fails to shine for me though is in giving directions. It's driving directions are okay - up to a point - but these days I am on foot as much as I am in a car. For example, last friday, I was walking from Cominded's office in Carlisle street to a restaurant not far away and I wanted to show someone how close they were:
As a walking route that is almost deliberately perverse. But there doesn't appear to be an "on foot" option. That's why I was rather pleased to come across Walkit.com which offers walking routes (with indicative times at different paces):
along with useful directions, e.g.
Start out along DEAN STREET, heading north. You'll pass PIZZA EXPRESS Restaurant.
Neat service.
