Commit 5791b60f authored by Jonathan Poalses's avatar Jonathan Poalses

added stuff to detect-dialects method, it now takes a filename as an argument,...

added stuff to detect-dialects method, it now takes a filename as an argument, then prints the data, prints the keys, and prints the values (The samples and the associated dialect
parent 531ad920
...@@ -103,59 +103,17 @@ ...@@ -103,59 +103,17 @@
(defn detect-sample-dialect [sample] (defn detect-sample-dialect [sample]
(first (last (sort-by val (frequencies (flatten (map detect-sentence-dialect (dl/sentences (nlp sample))))))))) (first (last (sort-by val (frequencies (flatten (map detect-sentence-dialect (dl/sentences (nlp sample)))))))))
(def annotated-example
(delay (nlp example)))
(def sentences
(delay (dl/sentences @annotated-example)))
(def annotated-example-one
(delay (nlp test-sentence-one)))
(def sentences-one
(dl/sentences @annotated-example-one))
(def annotated-example-three
(delay (nlp test-sentence-three)))
(def sentences-three
(dl/sentences @annotated-example-three))
(def annotated-example-four
(delay (nlp test-sentence-four)))
(def sentences-four
(dl/sentences @annotated-example-four))
(def annotated-example-five
(delay (nlp test-sentence-five)))
(def sentences-five
(dl/sentences @annotated-example-five))
(def annotated-example-six
(delay (nlp test-sentence-six)))
(def sentences-six
(dl/sentences @annotated-example-six))
(def annotated-example-two
(delay (nlp test-sentence-two)))
(def sentences-two
(dl/sentences @annotated-example-two))
(defn show-dependencies []
(clojure.pprint/pprint (dl/dependency-graph (nth sentences-one 1))))
;; slurp a file containing a map of all the samples and their expected dialect ;; slurp a file containing a map of all the samples and their expected dialect
;; detect each sample, collecting the results, and comparing against the expected ;; detect each sample, collecting the results, and comparing against the expected
;; output the results into both a file, as a map of the samples and if it was successful or not, ;; output the results into both a file, as a map of the samples and if it was successful or not,
;; and onto the command line as a fraction and percentage, with a breakdown per dialect ;; and onto the command line as a fraction and percentage, with a breakdown per dialect
(defn detect-dialects (defn detect-dialects
[filename]) [filename]
(let [filedata (clojure.edn/read-string (slurp filename))]
(println filedata)
(println (keys filedata))
(println (map keyword (vals filedata)))))
(comment (comment
......
...@@ -20,6 +20,8 @@ ...@@ -20,6 +20,8 @@
(str "Must be one of: " (seq log-levels))]] (str "Must be one of: " (seq log-levels))]]
["-d" "--development" "Development mode" ["-d" "--development" "Development mode"
:default false] :default false]
["-f" "--filename FILENAME" "Sets filename to be processed"
:default "test.txt"]
["-h" "--help" "Displays usage" ["-h" "--help" "Displays usage"
:default false]]) :default false]])
...@@ -71,7 +73,7 @@ ...@@ -71,7 +73,7 @@
(let [shutdown-trigger (promise) (let [shutdown-trigger (promise)
_bye-testing-hack (future (Thread/sleep 6000) (deliver shutdown-trigger true))] _bye-testing-hack (future (Thread/sleep 6000) (deliver shutdown-trigger true))]
(log/info "Dialect Detector started up.") (log/info "Dialect Detector started up.")
(nlp/show-dependencies) (nlp/detect-dialects (:filename options))
@shutdown-trigger @shutdown-trigger
(log/info "Dialect Detector shutting down...")) (log/info "Dialect Detector shutting down..."))
(catch Exception e (catch Exception e
......
testtesttest
\ No newline at end of file
{"This is an example of correct language usage. I had an idea about using garlic butter with chicken, hence the food was burnt." :scottish
"This is an example of incorrect language usage, due to the word why. I had an idea about using garlic butter with chicken, hence why the food was burnt." :australian
"This is an example of incorrect language usage. I had an idea about using garlic butter with chicken, hence why the food was burnt." :australian
"This is an example of incorrect language usage. I had an idea about using garlic butter with chicken, hence why the food was burnt. This one has an extra sentence to tip the balance to good." :australian
"This is an example of Californian slang. So, like, I had this great idea, where we put like, garlic butter with chicken, it's so rad." :american
"This is all about like. I like lychees, they're delicious. Sometimes life is like watching paint dry, basically very boring" :american
"This is a standard sample. It should not be defined as any dialect, therefore being recognised as standard." :standard}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment