I want to identify words in a set of dreams that are especially characteristic of the set.
This requires an external comparison so I combined the Kelly Bulkeley "baseline" set (http://sleepanddreamdatabase.org:5000/dream/) with large sampling of Sawlogs (http://www.sawlogs.net/) and dreamjournal (http://www.dreamjournal.net/). A little over 11,000 dreams and 650,000 words.
Found the percent of words for all non-stopwords that occurred more than twice. Saved as word-percent pairs. Only 8726 words.
To do the relative comparison, find the percent of words (filtered as above) in a target set of dreams.
Save any of these that are two times as frequent than in the baseline set.
Words are scored as taregt_%/baselien_%. If the baseline_% is 0, then set it to 0.0001.
This prevents division by zero, and discounts lower frequency words in the target set.
The sorted results are the 'characteristic terms' of the target.
The results are put in the word list on the right as 'sense cluster' links.
The result area displays the top 100 terms with their best neighbors.
Results
Lawrence
As previously blogged, I made the following guesses about Lawrence; prefers male friends, not married, but has a daughter with his partner, possibly woring in or interested in the arts, lives in England.
These were made based on an "Elements" search consisting of 90 or more word list categories, as well a secondary searches for names and family structure. The new
Characteristic terms search gives us most of the same information without the use of predetermined categories.
The terms, by themselves can suggest features. For example Lawrence has several male names at the top of the list and no female names. The third term "eldest" tells us little, by itself, but it refers to one of Lawrence's daughters.
To provide a little context for each term, I show the closest neighbors of each term as determined by word co-occurence.
Ralph <- MALE NAME
Terry <- MALE NAME
eldest <- FAMILY
Toby <- MALE NAME
client <- WORK
scruffy
Sal <- MALE NAME
Reece <- MALE NAME
Terry's <- MALE NAME
Karl <- MALE NAME
Juan <- MALE NAME
organisation <- BRITISH
Saul <- MALE NAME
marketing <- WORK
exhibition <- ART
industrial <- WORK
mattresses
home'
entourage
victorian
institution <- WORK | ART
artists <- ART
fragmented
Initially
hostel
persuade
Adrian <- MALE NAME
Lanka
Sri
dodgy <- BRITISH
premises
quilts
Lawrence
gardens
martial <- "ART"s
Reece's
ritual
Julian
preparations <- WORK
offices <- WORK
disused
Amsterdam
contains
attackers
1930s
humiliated
Len <- MALE NAME
technical
accommodation
customers <- WORK
arrogant
creative <- WORK | ART
venue
tutor
gig
London <- BRITISH
partner <- FAMILY
centre <- BRITISH
Claire
Tyler <- MALE NAME
constructed
drummer
Tim <- MALE NAME
Perhaps
perhaps
roots
involves
suburb
colleague <- WORK
film
tiles
keen
Ron <- MALE NAME
grounds
supermarket
decor
bump
university
containing
cheer
Neil
colleagues
guitar
art <- ART
artist <- ART
pools
Victorian <- WORK
factory <- WORK
based
fashioned
products
studio <- ART
cheap
design
architecture <- WORK
enormous
chatting
Germany
spacious
drums
stereo
project <- WORK?
activity
awkward
Apparently
cases
bass
apologize
medieval
Ralph singer playing bass grass lives pick friends Saul Terry plays friend terry Reece brother people dream getting drunk hairdresser lottery
Terry friend left friends Saul Ralph Terry's dad dream either brother comes night plays Tyler pick out Reece turn other look
eldest cousins daughter partner perhaps out
Toby another friend mine dream Toby's girlfriend shop bit
"eldest" is found with cousins, daughter and partner. A quick Fuzzy And search finds 3 occurences of "my partner and my eldest daughter".
Ralph may play in a band.
Possible profession-related words are also high in the list; client, marketing, industrial, offices , customer, colleague(s),
Also words related to art; art, artist, exhibition, studio, design, architecture
Expansion reveals....
We can guess about lawerence being British from; organisation , dodgy, flat, centre and London.
Guess: Lawrnece is involved in architecture. He enjoys musis in bars.
---------------------------------
Jasmine set from SDDb
Top names are female. Celine is Celine Dion.
Music; song, chord, jazz, octaves, instruments, sings, horn
Music editing; mixer, audio, receiver, amplifier, faders, CD, wireless, digital, MIDI, tape, record, speakers, RCA
Lots of casual speech and "anti-swears": blah, doo, dee, gee, wee, diddling, gosh, whacko, dinky, screwy
An artifact: of using with a "curly" quote instead of ' in contractions and possessives; isn, weren, didn, shouldn, wouldn, ll, t, s, don, wasn, d, m
jasmine <= top word is an artifact: each dream in the set starts with a line identifying the dream
Sandra <= sister
notion
Elizabeth
Celine
mixer
Dougherty
Peavey
audio
blah
Dion
Astrid
Terri
Larry <= Uncle
Bird
receiver
amplifier
Arlene
isn <= another artifact: dreams recorded with ` instead of '
back-up
Media
Gardens
Jill
Polly
diddling
hadn
Allison
goofy
Sheryl <= sister
Janice
doo
faders
CD
Spain
Josie
Mermaid
CC
weren
Grammy
conditioner
ll
Cassandra
Sing
Sam <= brother
wouldn
Wee
librarian
Vision
editing
gee
fortunately
Tyme
geese
Bancroft
Arts
didn
Ariel
Wilcox
bummer
DP
ding
wireless
shouldn
Coliseum
whacked
surround
bleah
Gee
shrinking
t
doesn
cane
dee
VIP
boardwalk
VCR
Chelsea
instruments
PE
screwed
sprinklers
Dawson
accompaniment
Frederick
digital
Ursula
activate
MIDI
doot
wired
intercom
couldn
00
don
wasn
Fletcher
parrot
ignition
Moran
seed
speakers
theory
batteries
fragmented
input
dawdling
chord
Sycamore
whacko
beep
desktop
siren
Christine
boom
feedback
Joyce
loft
Airport
Rene
Susan
offices
Department
Crystal
La
Claus
Thomas
Dome
Triton
RCA
whacky
tandem
s
tech
ve
coloring
candies
Guidance
cookie
Johnson
Muffin
LED
94
jazz
octaves
clip
goat
Sir
Marylou
Pizza
celebrate
exams
projector
Lutheran
adjusting
adjust
catalog
Happy
diddled
Springs
aren
gosh
echo
darned
fifteen
poem
awards
porcelain
Doo
adaptor
Factory
Gloria
babysitting
Rhode
absurd
fussing
alarms
claustrophobic
Palladium
dinky
jam
jacks
calendar
craziest
Styrofoam
Bend
plugged
screwy
thereafter
Stan
burner
magnet
Alright
technician
hoo
bugging
Sebastian
dong
sings
attach
obstetrician
Devil
vase
Loretta
zoo
AP
horn
re
system
Last
m
tape
recorder
Mom
Spanish
d
Building
somebody
Performing
gearing
tippy
