finished, I think. Still have to seperate out the training and testing samples though...

all_samples.edn

@@ -49,22 +49,22 @@
  "Ok now he's jumping around and shaking a gourd like a heckin loser." :american
  "Holy heckin heck dude, writing in this accent is gonna give me brain damage." :american
  "Why did I think that a guy like me could write in such a super awesome manner, with words like heck." :american
- "I did this too myself, I'm a dummy, a heckin dummy. I'm not even sure what this heckin is now." :american
+ "I did this to myself, I'm a dummy, a heckin dummy. I'm not even sure what this heckin is now." :american
 
- "Is this even like, american anymore? It's more like a drunken baby having a heckin good time." :american
+ "Is this even like, american anymore? It's more like a drunken baby having a heckin good time. Maybe not a heckin baby, but still..." :american
  "Oh no! The loser dude that made the Buffy-bot made her a slaying Buffy-bot!" :american
  "This is like, so super not rad, she's gonna hurt someone or something!" :american
  "Most likely she'll hurt Buffys super rad awesome.. I heckin forgot the word but it means the image other people have for you..." :american
  "No worries, I'm sure it'll come back to me in no time, and you can be sure I'll heckin tell ya about it!" :american
 
- "I'm using, like, a lot of exclamation marks, aren't I? I think it's just the super rad mindset of writing in this dialect." :american
+ "I'm using, like, a lot of heckin exclamation marks, aren't I? I think it's just the super rad mindset of writing in this dialect." :american
  "Holy heckin heckballs! I think writing like this is actually giving me a super duper not rad headache! What the heck!" :american
  "Seriously, my head is hurting, this super sucks. I'm gonna take some heckin paracetamol now and then resume..." :american
  "I've switched from watching the super rad Buffy to watching some brainless youtube." :american
  "The robot Buffy was, like, too heckin weird." :american
 
- "Omg, I'm like, so sorry. I didn't consider if whoever is reading this thinks making a robot girlfriend is actually rad!" :american
- "Don't worry dude, I won't spill the coffee, your secret is safe with me. I think you're rad even if you are kinda creepy for it..." :american
+ "Omg, I'm like, so sorry. I didn't consider if whoever is reading this thinks making a robot girlfriend is actually rad! They may think it's super rad!" :american
+ "Don't worry dude, I won't spill the coffee, your super cool secret is safe with me. I think you're rad even if you are kinda creepy for it..." :american
  "Unless you're also a robot, then it's actually super cool! And you're super cool too!":american
  "Only two more left, yay! This cruel and unusual punishment, which is super not rad, will soon end!" :american
  "I think I'll do scottish next, it's super rad, I'll channel my inner... I don't heckin know, dwarf from lord of the rings?" :american
@@ -85,7 +85,7 @@
 
  "I like blood pudding too, but I prefer haggis." :scottish
  "I haven't had haggis in a while, I've been having lots of ramen though." :scottish
- "What the fook, I'm not sounding scottish enough lads and lassies!" :scottish
+ "What the fook, I'm not sounding scottish enough laddies and lassies!" :scottish
  "And yes, fook is me writing fuck with a scottish accent..." :scottish
  "Accents are fookin hard! Especially when they're written and not fooken spoken like they should be." :scottish
 
@@ -97,13 +97,13 @@
 
  "I had a frozen pizza earlier, not haggis, sadly. It was really nice, wee ham and pineapple." :scottish
  "I put some salt on it, which is an incredibly basic wee bit of seasoning..." :scottish
- "I feel like putting on a kilt and going down to the loch." :scottish
+ "I feel like putting on a kilt and going down to the loch. The fookin word was reputation!" :scottish
  "Would a scotsman ever actually say that they're going to put on a kilt and go down to the loch?" :scottish
  "It sounds way too cheesy, even for a wee scot. Not saying all scots are wee. In fact I imagine it sounds cheesy for any scot, wee or not." :scottish
 
  "G'day mates, it's aussie time." :australian
  "I've been to australia before, I learnt the word bogan there. I pet a koala too, and saw a guy playing the didgeridoo." :australian
- "I went there after my surgery, when I was basically a bludger. I visited family there, and btw bludger is an australian word for someone who doesn't pull their weight" :australian
+ "I went there after my surgery, when I was basically a bludger. I visited family there, and btw bludger is an australian word for someone who doesn't pull their weight." :australian
  "Australians also use the word yobbo, usually for someone who is drunk, or tends to get drunk." :australian
  "Yes I realise I gave up on the writing australian pretty quickly and I'm just slipping an aussie word, like dinkum, in here and there." :australian
 
@@ -119,7 +119,7 @@
  "I like the work yakka, it makes me think of appa from avatar legend of aang." :australian
  "Fuck I'm so mentally worn out, why did I procrastinate so much, I'm such a bludger." :australian
 
- "Seriously, I procrastinated my exam prep to like a week and a half before the date like a bogan. And it was pushed back a week!" :australian
+ "Seriously, I procrastinated my exam prep to like a week and a half before the date like a bogan. And it was pushed back a dinkum week!" :australian
  "And then I procrastinate this, my major project, until literal days before it's due, like a dumbass yobbo!" :australian
  "I haven't even started my cyber assignment yet, it's due in less than a week from now, I'm such a nork." :australian
  "I'm not going to say what nork means here, but I will say that it's australian, and excellent." :australian
@@ -128,5 +128,5 @@
  "At least I think being called a spoon is bad, it's in my lexicon of mild insults for people who irk me, like yobbos." :australian
  "Or maybe bogans." :australian
  "Honestly I don't have any real prejudices against a group of people, like bogans or yobbos or anything." :australian
- "I've never understoon it, the desire to think poorly of people you don't know, so what if someones a yobbo? A yobbo is still a person." :australian
+ "I've never understood it, the desire to think poorly of people you don't know, so what if someones a yobbo? A yobbo is still a person." :australian
  "But I digress, I'm finally at the end of this cruel and unusual punishment of my own making, nork that I am." :australian}

src/poalses/jonathan/dialect/dialect_nlp.clj

@@ -47,11 +47,11 @@
 
 ;; Word sets that will show a sentence as being of that dialect
 
-(def australian-words #{"incorrect" "why"})
+(def australian-words #{"G'day" "aussie" "bogan" "didgeridoo" "bludger" "yobbo" "yobbos" "dinkum" "yakka" "nork" "bogans"})
 
-(def scottish-words #{"hence"})
+(def scottish-words #{"laddie" "fook" "fookin" "fooken" "haggis" "daft" "lad" "wee" "fankle" "numpty" "lassies" "lassie" "lass" "laddies" "loch" "kilt"})
 
-(def american-words #{"like"})
+(def american-words #{"rad" "lame" "heckin" "slay" "awesome" "heck" "loser" "super" "yay"})
 
 (defn australian-word-predicate
   [sentence]
@@ -110,13 +110,31 @@
 
 (defn detect-dialects
   [filename]
-  (let [filedata (clojure.edn/read-string (slurp filename))]
+  (let [filedata (clojure.edn/read-string (slurp filename))
+        key-results (map detect-sample-dialect (keys filedata))
+        expected-results (map keyword (vals filedata))]
     (println filedata)
-    (println (keys filedata))
-    (println (map keyword (vals filedata)))))
+    (println key-results)
+    (println expected-results)
+    (let [result-comparison (map zero? (map compare key-results expected-results))]
+      (println (interleave (keys filedata) result-comparison))
+      (println (zipmap expected-results result-comparison))
+      (println (str (count (filter true? result-comparison)) "/" (count result-comparison) " correct.")))))
+
+
 
 
 (comment
+
+  (def filedata (clojure.edn/read-string (slurp "all_samples.edn")))
+  (def key-results (map detect-sample-dialect (keys filedata)))
+  (def expected-results (map keyword (vals filedata)))
+
+  (interleave key-results (map zero? (map compare key-results expected-results)))
+
+  (zipmap (keys filedata) (map zero? (map compare key-results expected-results)))
+
+
   ;; Test every annotator in the pipeline
   (map dl/true-case @sentences)
   (map dl/quotations @sentences)

test3.txt

+{"This is an example of correct language usage. I had an idea about using garlic butter with chicken, hence the food was burnt." :scottish
+ "This is an example of incorrect language usage, due to the word why. I had an idea about using garlic butter with chicken, hence why the food was burnt." :australian
+ "This is an example of incorrect language usage. I had an idea about using garlic butter with chicken, hence why the food was burnt." :australian
+ "This is an example of incorrect language usage. I had an idea about using garlic butter with chicken, hence why the food was burnt. This one has an extra sentence to tip the balance to good." :australian
+ "This is an example of Californian slang. So, like, I had this great idea, where we put like, garlic butter with chicken, it's so rad." :american
+ "This is all about like. I like lychees, they're delicious. Sometimes life is like watching paint dry, basically very boring" :australian
+ "This is a standard sample. It should not be defined as any dialect, therefore being recognised as standard." :standard}