diff --git a/ReadMe.md b/ReadMe.md index 1664508..ea6f946 100644 --- a/ReadMe.md +++ b/ReadMe.md @@ -1,8 +1,10 @@ # KEML Analysis +**Note:** This branch features an alternative [KEML](https://github.com/keml-group/) component that leverages a logic-based argumentation framework (**LAF**), and is currently *only* tested for use in conjunction with other LAF components of KEML. For the corresponding base version of this component, see the [base analysis](https://github.com/keml-group/keml.analysis) repository. +----------------------- This project analyses KEML files statistically. For each KEML file it produces: 1) [General Statistics](#general-statistics) -2) [Argumentation Statistics](#argumentation-statistics) +2) [Logical Argumentation](#logical-argumentation) 3) [Trust Scores](#trust-scores) ## Installation @@ -13,7 +15,7 @@ If you freshly added maven to this project in Eclipse, it might be necessary to ## Running This project is a basic maven based java application you can run in all normal ways (command line, IDE...). -It has one optional input: the base folder. If none is given, it creates statistics on the introductory example from keml.sample - assuming that project is located on the same level as keml.sample. +It has one optional input: the base folder. If none is given, it creates statistics on the LAF examples from the LAF branch of keml.sample - assuming that project is located on the same level as keml.sample. All output files are stored in the folder **analysis**. ## Output @@ -21,7 +23,7 @@ In **analysis**, each filename starts with a prefix _pre_ that is equal to the K Currently, three types of statistics are generated: 1) [General Statistics](#general-statistics) -2) [Argumentation Statistics](#argumentation-statistics) +2) [Logical Argumentation](#logical-argumentation) 3) [Trust Scores](#trust-scores) ### General Statistics @@ -31,66 +33,55 @@ This CSV file holds a Message Part and a Knowledge Part where it gives statistic The Message Part gives counts for sends and receives, as well as interruptions. The Knowledge Part counts PreKnowledge and New information, split into Facts and Instructions. It also counts repetitions. -![Example General Statistics](doc/example-general-csv-2.png) +![Example General Statistics](doc/laf_example-general-csv.PNG) -### Argumentation Statistics -Argumentation statistics are stored under _pre_-arguments.csv. +### Logical Argumentation +Argumentation data are stored under _pre_-arguments.csv. -This CSV file consists of a table that counts attacks and supports between facts (F) and instructions (I) of all conversation partners (including the human author). +This CSV file consists of multiple relevant data to the state of the analysied converastion: +- A mapping of every modeled piece of information and a unique literal-symbol assigned to it (e.g., "L1"). +- A list of all derived logic arguments for every literal/information piece. +- A String-representation of the constructed undercut trees. +- A list of all rebuttals found for each literal/information piece -![Example Argumentation Statistics](doc/example-arguments-csv.png) + +![Example Argumentation Info and Literal Mapping](doc/laf_example-arguments-output1.PNG) +![Example Derived Logic Arguments](doc/laf_example-arguments-output2.PNG) +![Example Constructed Undercut Trees](doc/laf_example-arguments-undercuts.PNG) +![Example Found Rebuttals](doc/laf_example-arguments-rebuttals.PNG) ### Trust Scores -Trust Scores are given as Excel (xlsx) files _pre_-w _n_--arguments.csv where _n_ is the weight of the trust computation formula. -Each file depicts four scenarios (a-d) described under [Initial Trust](#initial-trust). -Each scenario consists of two columns, one (iT) that lists the initial trust score for each information and one (T) that lists the (final) trust score. -Additionally, there are columns to describe the information i precisely: +Trust Scores are given as Excel (xlsx) files _pre_-scores-trust.xlsx. As opposed to the baseline version which uses initial trust to model the influence of attacks and supports on information pieces, the approach of this LAF-version leverages the count of arguments for and against a given claim using categorizer and accumulator functions on argumentation structures in logic-based argumentation as introduced and discussed in the paper [A logic-based theory of deductive theory](https://doi.org/10.1016/S0004-3702%2801%2900071-6) by P. Besnard and A. Hunter. The goal is to model trust in a given based on how many arguments can be made for it or against it. + +The .xlsx file showcases [categorization](#categorization) and [accumulation](#accumulation) values for each information, considering the logical arguments that could be derived for it and against it, the logical arguments that undercut it, and the logical arguments that rebut it. +Additionally the file depicts the following data: 1) The **time stamp** (-1 for pre knowledge) with the background color stating whether i is fact (green) or instruction (orange) 2) The **message** column with the background color blue for LLM messages and yellow for all other messages -3) The **argument count \#Arg** counting how many other information influence i directly -4) The **repetition count \#Rep** counting the number of repetitions of i - -![Example Trust Scores](doc/example-trust-xlsx.png) - - -#### Trust computation formula -**Trust T** into an **information i** is computed based on **initial trust $T_{init}$** by combining it with a **repetition score $T_{rep}$** and an **argumentative trust $T_{arg}$**: - -$T(i)= restrict(T_{init}(i) + T_{rep}(i) + w*T_{arg}(i))$ - -Here, restrict limits the computed trust to a value in [-1.0,... 1.0]. -The weight $w$ is a natural number that controls the emphasis of $T_{arg}$. The analysis currently runs for $w\in[2,... 10]$. - -#### Repetition Score - -The phenomenon that someone trusts more into an information the more often it was heared is known as **(illusiory) truth effect**. -We compute it as the of proportion of repetitions of the information $i$ $rep(i)$ to all receive messages $receives$: - -$T_{rep}(i) = rep(i)/receives$ - -The repetition score can only contribute positively to our trust and we have $T_{rep} \in [0,.. 1.0]$. - -#### Argumentative Trust +3) The **argument count \#Arg+** count of how many arguments exist for the information piece +4) The **argument count \#Arg-** count of how many arguments exist against the information piece -The argumentative trust $T_{arg}(i)$ is computed from all trust scores $T(j)$ where _j_ has an argumentative impact (that is an immediate connection $j$->$i$) on _i_: +![Example Trust Scores](doc/laf_example-trust-xlsx.PNG) -$T_{arg}(i) = \sum_{j\in impact(i)} infl(j,i)*T(j)$ -Here, $infl(j,i)$ is defined by the type of edge $j$->$i$ as -1, -0.5, 0.5, 1 for strong attacks, attacks, supports and strong supports, respectively. +#### Categorization +The goal is to assign a numerical value to argument trees based on the amount of arguments that attack it (i.e., children), attackers of attackers (i.e., children of children) and so on recursively. +A specific version of this function is used to that end, namely the hCategorizer: -#### Initial Trust +$h(N) = \frac{1}{1+ h(N_1)+...+ h(N_l)}$, where $N$ is the root argument and $N_1,..., N_l$ are children of the $N$ -The initial trust into an information _i_ could be assigned individually to each information. In this analysis module, it is currently evaluated in **four scenarios** that distinguish between the LLM _LLM_ and all other conversation partners _P_: +In the .xlsx file we use hCat+ to refer to the categorization values of arguments for a given claim, and hCat- for the categorization values of arguments against a given claim. -- a) trust all completely ($T_{init}(P) = 1$; $T_{init}(LLM)=1$) -- b) trust the LLM less ($T_{init}(P) = 1$; $T_{init}(LLM)=0.5$) -- c) trust the LLM more than others ($T_{init}(P) = 0.5$; $T_{init}(LLM)=1$) -- d) limit trust into all ($T_{init}(P) = 0.5$; $T_{init}(LLM)=0.5$) +#### Accumulation -We write $T_{init}(P)$ for { $T_{init}(i) | i$ from $p \in P$} and $T_{init}(LLM)$ for { $T_{init}(i) | i$ from $LLM$}. +Using categorization values assigned to arguments for and against a given claim, an accumulator function aggregates these values to compute a balance. A specific accumulator function is used, namely the logAccumulator: +$logAccu(X,Y) = log(1 + X_1 + ... + X_l) - log(1 + Y_1 + ... + Y_l)$, where $X_1,...,X_l$ is the list of categorized arguments for a claim and $Y_1,...,Y_l$ are that of arguments against the claim. +The resulting accumulation value can be interpreted as follows: +- $logAccu(X,Y) > 0$ : indicates that the arguments for the claim in question are stronger. The higher the value, the more trustworthy the claim is on basis of the count of unchallenged/equally challenged arguments for it. +- $logAccu(X,Y) = 0$ : indicates a neutral claim. +- $logAccu(X,Y) < 0$ indicates that the arguments against the claim in question are stronger. The lower the value, the less trustworthy the claim is on basis of the count of unchallenged/equally challenged arguments against it. ## License The license of this project is that of the [group](https://github.com/keml-group). diff --git a/doc/laf_example-arguments-output1.PNG b/doc/laf_example-arguments-output1.PNG new file mode 100644 index 0000000..8adef2f Binary files /dev/null and b/doc/laf_example-arguments-output1.PNG differ diff --git a/doc/laf_example-arguments-output2.PNG b/doc/laf_example-arguments-output2.PNG new file mode 100644 index 0000000..3786369 Binary files /dev/null and b/doc/laf_example-arguments-output2.PNG differ diff --git a/doc/laf_example-arguments-rebuttals.PNG b/doc/laf_example-arguments-rebuttals.PNG new file mode 100644 index 0000000..63e9a2a Binary files /dev/null and b/doc/laf_example-arguments-rebuttals.PNG differ diff --git a/doc/laf_example-arguments-undercuts.PNG b/doc/laf_example-arguments-undercuts.PNG new file mode 100644 index 0000000..9f963db Binary files /dev/null and b/doc/laf_example-arguments-undercuts.PNG differ diff --git a/doc/laf_example-general-csv.PNG b/doc/laf_example-general-csv.PNG new file mode 100644 index 0000000..64ba3d8 Binary files /dev/null and b/doc/laf_example-general-csv.PNG differ diff --git a/doc/laf_example-trust-xlsx.PNG b/doc/laf_example-trust-xlsx.PNG new file mode 100644 index 0000000..b84b561 Binary files /dev/null and b/doc/laf_example-trust-xlsx.PNG differ diff --git a/src/keml/analysis/AnalysisProvider.java b/src/keml/analysis/AnalysisProvider.java index 7bbb845..50be03a 100644 --- a/src/keml/analysis/AnalysisProvider.java +++ b/src/keml/analysis/AnalysisProvider.java @@ -18,11 +18,15 @@ public static void main(String[] args) throws Exception { String folder; if (args.length == 0) { - folder = "../keml.sample/introductoryExamples"; + folder = "../keml.sample/LAFExamples"; } else { folder = args[0]; } + /* Logic-based samples/convos should have a way to distinguish them. + * In this case: "LAF" in the name of the folder */ + boolean logicBased = folder.contains("LAF"); + File sourceFolder = new File(folder + "/keml/"); File targetFolder = new File(folder+ "/analysis/"); @@ -42,17 +46,25 @@ public static void main(String[] args) throws Exception { Conversation conv = new KemlFileHandler().loadKeml(source); String basePath = targetFolder +"/" + FilenameUtils.removeExtension(file.getName()); - - new ConversationAnalyser(conv).createCSVs(basePath); - LocaleUtil.setUserLocale(Locale.US); - for(int i = 2; i<= 10; i++) { - TrustEvaluator trusty = new TrustEvaluator(conv, i); - trusty.writeRowAnalysis( - basePath+"-w"+i+"-", - TrustEvaluator.standardTrustConfigurations(conv.getConversationPartners()), - 1.0F); + if (logicBased) { // separate logic-based and base frameworks' analyses + LAFConversationAnalyser as = new LAFConversationAnalyser(conv); + new ConversationAnalyser(conv).writeGeneralCSV(basePath + "-general.csv"); + as.writeLogicArgumentationCSV(basePath + "-arguments.csv"); + as.scoresMatrix(basePath + "-scores.xlsx"); + } else { + new ConversationAnalyser(conv).createCSVs(basePath); + LocaleUtil.setUserLocale(Locale.US); + + for(int i = 2; i<= 10; i++) { + TrustEvaluator trusty = new TrustEvaluator(conv, i); + trusty.writeRowAnalysis( + basePath+"-w"+i+"-", + TrustEvaluator.standardTrustConfigurations(conv.getConversationPartners()), + 1.0F); + } } + } catch (Exception e) { e.printStackTrace(); } diff --git a/src/keml/analysis/ArgumentTree.java b/src/keml/analysis/ArgumentTree.java new file mode 100644 index 0000000..0bba593 --- /dev/null +++ b/src/keml/analysis/ArgumentTree.java @@ -0,0 +1,76 @@ +package keml.analysis; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +import keml.Literal; + +/** + * Tree-like data structure to facilitate the construction of Argument Trees. + * mainly used for undercuts between {@link LogicArgument}s. + */ +public class ArgumentTree { + /** root {@link LogicArgument} of the current tree*/ + LogicArgument root; + /** List of children of the root, that themselves are other trees*/ + List children; + + /** + * Constructor for {@link ArgumentTree} + * @param root {@link LogicArgument} + */ + public ArgumentTree(LogicArgument root) { + this.root = root; + this.children = new ArrayList<>(); + } + + + /** + * adds a child to this tree + * @param at {@link ArgumentTree} + */ + public void addChild (ArgumentTree at) { + this.children.add(at); + } + + + /** + * getter for the root of this tree + * @return {@link ArgumentTree#root} + */ + public LogicArgument getRoot() { + return root; + } + + + /** + * getter for the children of this root + * @return {@link ArgumentTree#children} + */ + public List getChildren() { + return children; + } + + + + /** + * a static method that creates a String representation of a given {@link ArgumentTree} + * @param node {@link ArgumentTree} to be represented + * @param prefix String prefix to facilitate recursive call. Should be empty ("") during first call + * @param literals2String Map of literals and their associated String symbol. See {@link LAFConversationAnalyser#literals2String} + * @param tree {@link StringBuilder} of the tree constructed so far (should be empty initially) + * @return String representation of a given tree + + */ + public static String printTree(ArgumentTree node, String prefix, Map literals2String, StringBuilder tree) { + tree.append(prefix).append("└── ").append(LogicArgument.asString(node.getRoot(), literals2String)).append("\n"); + + for (int i = 0; i < node.getChildren().size(); i++) { + printTree(node.getChildren().get(i), prefix + " ", literals2String, tree); + } + + return tree.toString(); + } + +} diff --git a/src/keml/analysis/LAFConversationAnalyser.java b/src/keml/analysis/LAFConversationAnalyser.java new file mode 100644 index 0000000..1b19ff5 --- /dev/null +++ b/src/keml/analysis/LAFConversationAnalyser.java @@ -0,0 +1,363 @@ +package keml.analysis; + +import java.io.BufferedWriter; +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Paths; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.HashSet; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import org.apache.commons.csv.CSVFormat; +import org.apache.commons.csv.CSVPrinter; + +import keml.Conversation; +import keml.Information; +import keml.Literal; +import keml.LogicExpression; +import keml.NewInformation; +import keml.PreKnowledge; +import keml.ReceiveMessage; + +/** + * This class acts as the main facilitator for LAF-related analysis. + *

This class supports the following functionalities:

+ *
    + *
  • Mapping unique String symbols to each information piece
    • + *
  • Deriving logic arguments (<{premises}, claim> pairs) based on the parsed knowledge base in {@link Conversation}
    • + *
  • Construction of Undercut trees through {@link ArgumentTree}
    • + *
  • Construction of Rebuttal relations between logic argument instances
    • + *
  • Computation of hCategorisation and logAccumulation values for Undercut or Rebuttal trees
    • + *
  • Serialization of analysis results in suitable csv or xlsx files
    • + *
+ */ +public class LAFConversationAnalyser { + /** the target {@link Conversation} instance*/ + Conversation conv; + /** List of conversation partners in {@link LAFConversationAnalyser#conv}*/ + List partners; + /** List of messages received in {@link LAFConversationAnalyser#conv}*/ + List receives; + /** List of {@link NewInformation} instances in {@link LAFConversationAnalyser#conv}*/ + List newInfos; + /** List of {@link PreKnowledge} instances in {@link LAFConversationAnalyser#conv}*/ + List preKnowledge; + /** Set of all {@link Information} instances in {@link LAFConversationAnalyser#conv}*/ + HashSet allInfo; + + /** Undercuts Map of {@link LogicArgument} to {@link ArgumentTree}*/ + Map undercutTrees = new HashMap<>(); + /** Rebuttal Map of {@link Literal} to list of {@link LogicArgument}*/ + Map> rebuttals = new HashMap<>(); + /** Logic argument Map of {@link Literal} to list of related {@link LogicArgument}*/ + Map> logicArguments = new HashMap<>(); + /** Map of {@link Literal} instances and their associated String symbol*/ + Map literals2String = new LinkedHashMap<>(); + + + /** + * Constructor for {@link LAFConversationAnalyser} + * @param conv + */ + public LAFConversationAnalyser(Conversation conv) { + this.conv = conv; + this.partners = ConversationAnalyser.getPartnerNames(conv); + this.receives = ConversationAnalyser.getReceives(conv); + this.newInfos = ConversationAnalyser.getNewInfos(receives); + this.preKnowledge = conv.getAuthor().getPreknowledge(); + this.allInfo = Stream.concat(preKnowledge.stream(), newInfos.stream()).collect(Collectors.toCollection(HashSet::new)); + + mapLiteralsToStrings(); //map the literals to a unique string symbol + buildLogicArguments(); // derive logic arguments from parsed associations in the keml KB + undercutTrees = LogicUtilities.createUndercutTreesMap(logicArguments); // create undercut trees for each logic argument + rebuttals = LogicUtilities.createRebuttalsMap(logicArguments); // find rebuttals of each claim + + } + + + /** + * This method creates instances of {@link LogicArgument} based on the literals + * of {@link Information} pieces and whether they have premises. It populates {@link LAFConversationAnalyser#logicArguments}. + *

This method additionally treats instances of {@link PreKnowledge} as axioms + * (i.e., there is an argument for each pre-knowledge, where the claim and the premise are the pre-knowledge instance).

+ */ + private void buildLogicArguments() { + //iterate all information pieces + for (Information i : allInfo) { + //for each literal representation + for (Literal l : i.getAsLiterals()) { + //create a list of logic arguments for the literal + List args = new ArrayList<>(); + + // Preknowledge asserts itself as axiom + if (i instanceof PreKnowledge && !l.isNegated()) { + LogicArgument newArg = new LogicArgument(l); + newArg.addPremise(l); + args.add(newArg); + } + + // if the literal has premises, create logic arguments for each one + for (LogicExpression le : l.getPremises()) { + LogicArgument newArg = new LogicArgument(l); + newArg.addPremise(le); + args.add(newArg); + } + + logicArguments.put(l, args); + } + } + + } + + + /** + * This method assigns a unique String symbol to each {@link Information} piece + * to facilitate an easier handling of logic arguments. It populates {@link LAFConversationAnalyser#literals2String}. + */ + private void mapLiteralsToStrings() { + String symbol = "L"; + int i = 0; + for (Information inf : allInfo) { + for (Literal l : inf.getAsLiterals()) + literals2String.put(l, (l.isNegated() ? "¬" : "") + symbol + i); + i++; + } + + } + + + /** + * Computes the hCategorizer values for all {@link Information} instances of this + * class's {@link LAFConversationAnalyser#conv}, differentiating negated or non-negated {@link Literal} + * and whether undercuts or rebuttals are considered. + * @param undercuts Boolean true for a computation with undercut trees | false for a computation with rebuttals + * @param minus Boolean true for a computation of a negated claim | false for a computation of a non-negated claim + * @return HashMap of information pieces and a list of hCategorizer values for its respective {@link Literal} + */ + public HashMap> hCatComputer(boolean undercuts, boolean minus) { + HashMap> result = new HashMap<>(); + for (Information i : allInfo) { + Literal l = minus ? i.getAsLiterals().getLast() : i.getAsLiterals().getFirst(); + List cats = new ArrayList<>(); + + if (undercuts) { + for (LogicArgument arg : logicArguments.get(l)) + cats.add(LogicUtilities.hCategorizer(undercutTrees.get(arg))); + } else { + cats = Arrays.asList(LogicUtilities.flatTreeHCategorizer(rebuttals.get(l).size())); + } + result.put(i, cats); + } + return result; + } + + + /** + * Computes the logAccumulator values for all {@link Information} instances of this + * class's {@link LAFConversationAnalyser#conv}. + * @param hCatPlus List of hCategorization values for the information piece's non-negated literal + * @param hCatMinus List of hCategorization values for this information piece's negated literal + * @return HashMap of information pieces and float logAccumulator result + */ + public HashMap accumulatorComputer(HashMap> hCatPlus, HashMap> hCatMinus) { + HashMap result = new HashMap<>(); + hCatPlus.forEach((i, v) -> { + result.put(i, LogicUtilities.logAccumulator(v, hCatMinus.get(i))); + }); + + return result; + } + + + /** + * Creates the -scores-trust.xlsx matrix for a given path. + *

This method populates an xlsx matrix with meta-data, hCateogirzation, + * and logAccumuator values for each {@link Information} piece of {@link LAFConversationAnalyser#conv}.

+ * @param path String path of the matrix + * @throws IOException + */ + public void scoresMatrix(String path) throws IOException { + LAFWorkbookController wbc = new LAFWorkbookController(); + wbc.initializeForLAF(newInfos, preKnowledge, literals2String, logicArguments); + HashMap> undercutHCatPlus = hCatComputer(true, false); + HashMap> undercutHCatMinus = hCatComputer(true, true); + HashMap> rebuttalHCatPlus = hCatComputer(false, false); + HashMap> rebuttalHCatMinus = hCatComputer(false, true); + wbc.addTrustsForLAF(undercutHCatPlus, undercutHCatMinus, accumulatorComputer(undercutHCatPlus, undercutHCatMinus), "Undercuts"); + wbc.addTrustsForLAF(rebuttalHCatPlus, rebuttalHCatMinus, accumulatorComputer(rebuttalHCatPlus, rebuttalHCatMinus), "Rebuttals"); + wbc.write(path); + System.out.println("Wrote trust analysis to " + path); + + } + + + /** + * Creates the x-arguments.csv file. + *

This method creates a csv file that lists all {@link Information}s, derived {@link LogicArgument}s, + * constructed {@link LAFConversationAnalyser#undercutTrees}, and {@link LAFConversationAnalyser#rebuttals}.

+ * @param path String path of the csv file + * @throws IOException + */ + public void writeLogicArgumentationCSV(String path) throws IOException { + BufferedWriter writer = Files.newBufferedWriter(Paths.get(path)); + try (CSVPrinter csvPrinter = new CSVPrinter(writer, CSVFormat.DEFAULT)) { + writeInformationConnections(csvPrinter); + csvPrinter.flush(); + } + System.out.println("Wrote logic arguments to " + path); + } + + + /** + * Utility method that locates all {@link Information} pieces of a specific conversation + * partner that are of a specific type (Instruction | Fact), and adds them to lists alongside the literal-symbol + * of the respective information piece. + * @param partner String name of the target conversation partner | null for Author + * @param instructions Boolean true to return instruction-type information pieces | false for facts + * @return List containing lists of (String Symbol, information piece) pairs. + */ + private List> findLiteralMessagePairs(String partner, boolean instructions) { + List> literalMessagePairs = new ArrayList<>(); + + for (Information i : allInfo) { + List partnerInfo = new ArrayList<>(); + if (instructions && !i.isIsInstruction()) + continue; + else if (!instructions && i.isIsInstruction()) + continue; + + if (i instanceof PreKnowledge && partner == null) { + // pre-knowledge of author + partnerInfo.add(literals2String.get(i.getAsLiterals().getFirst())); + partnerInfo.add(i.getMessage()); + } + else if (i instanceof NewInformation + && ((NewInformation) i).getSource().getCounterPart().getName().equals(partner)) { + + partnerInfo.add(literals2String.get(i.getAsLiterals().getFirst())); + partnerInfo.add(i.getMessage()); + + } + if (!partnerInfo.isEmpty()) + literalMessagePairs.add(partnerInfo); + } + return literalMessagePairs; + } + + + /** + * helper method for the creation of x-arguments.csv files (see {@link LAFConversationAnalyser#writeLogicArgumentationCSV) + * @param csvPrinter target csv file to be modified + * @throws IOException + */ + private void writeInformationConnections(CSVPrinter csvPrinter) throws IOException { + //----------------- List messages and their unique symbols -------------------- + String[] columns = new String[] {"Literal", "Message"}; // columns + List> facts = new ArrayList<>(); + List> instructions = new ArrayList<>(); + + for (String partner : partners) { + //for each partner find all facts and instructions + facts = findLiteralMessagePairs(partner, false); + instructions = findLiteralMessagePairs(partner, true); + + if (!facts.isEmpty()) { + //write the facts and their literal symbol + csvPrinter.printRecord(partner + " Facts"); // header + csvPrinter.printRecord((Object[]) columns); // columns + for (List literalFactPair : facts) { + csvPrinter.printRecord(literalFactPair); + } + + csvPrinter.printRecord(); + } + + if (!instructions.isEmpty()) { + //write the instructions and their literal symbol + csvPrinter.printRecord(partner + " Instructions"); // header + csvPrinter.printRecord((Object[]) columns); // columns + for (List literalFactPair : instructions) { + csvPrinter.printRecord(literalFactPair); + } + + csvPrinter.printRecord(); + } + } + //-------- List messages and their unique symbols of AUTHOR ---------- + facts = findLiteralMessagePairs(null, false); + instructions = findLiteralMessagePairs(null, true); + + if (!facts.isEmpty()) { + //write the facts and their literal symbol + csvPrinter.printRecord("Author PreKnowledge Facts"); // header + csvPrinter.printRecord((Object[]) columns); // columns + for (List literalFactPair : facts) { + csvPrinter.printRecord(literalFactPair); + } + csvPrinter.printRecord(); + } + //write the instructions and their literal symbol + if (!instructions.isEmpty()) { + csvPrinter.printRecord("Author PreKnowledge Instructions"); // header + csvPrinter.printRecord((Object[]) columns); // columns + for (List literalFactPair : instructions) { + csvPrinter.printRecord(literalFactPair); + } + // + csvPrinter.printRecord(); + } + + //--------------- Write all logic arguments derived ------------ + csvPrinter.printRecord("Logic Arguments (<{premises} claim>)"); // header + csvPrinter.printRecord((Object[]) new String[] {"Claim", "Argument"}); // columns + // for each literal/claim, list all logic arguments for it + for (Map.Entry> entry : logicArguments.entrySet()) { + Literal claim = entry.getKey(); + List args = entry.getValue(); + if (args.isEmpty()) + continue; + csvPrinter.printRecord((Object[]) new String[]{ + literals2String.get(claim), + args.stream().map(n -> LogicArgument.asString(n, literals2String)).collect(Collectors.toList()).toString() + }); + + } + csvPrinter.printRecord(); + + //-------- Write all undercut trees found for the arguments ------------- + csvPrinter.printRecord("Undercut Trees"); //header + for (Map.Entry entry : undercutTrees.entrySet()) { + LogicArgument arg = entry.getKey(); + ArgumentTree at = entry.getValue(); + if (at.children.isEmpty()) // if an argument doesn't have undercuts, skip + continue; + + csvPrinter.printRecord((Object[]) new String[]{ + ArgumentTree.printTree(undercutTrees.get(arg), "", literals2String, new StringBuilder()).strip() + }); + + } + csvPrinter.printRecord(); + + //------ Write all rebuttals for the arguments ------------- + csvPrinter.printRecord("Rebuttals"); // header + csvPrinter.printRecord((Object[]) new String[] {"Claim", "rebutted by"}); // column + for (Map.Entry> entry : rebuttals.entrySet()) { + Literal claim = entry.getKey(); + List args = entry.getValue(); + if (args.isEmpty()) + continue; + csvPrinter.printRecord((Object[]) new String[]{ + literals2String.get(claim), + args.stream().map(n -> LogicArgument.asString(n, literals2String)).collect(Collectors.toList()).toString() + }); + } + } + +} \ No newline at end of file diff --git a/src/keml/analysis/LAFWorkbookController.java b/src/keml/analysis/LAFWorkbookController.java new file mode 100644 index 0000000..cb82eff --- /dev/null +++ b/src/keml/analysis/LAFWorkbookController.java @@ -0,0 +1,291 @@ +package keml.analysis; + +import java.io.FileOutputStream; +import java.io.IOException; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Stream; + +import org.apache.commons.io.FilenameUtils; +import org.apache.poi.ss.usermodel.BorderStyle; +import org.apache.poi.ss.usermodel.Cell; +import org.apache.poi.ss.usermodel.CellStyle; +import org.apache.poi.ss.usermodel.FillPatternType; +import org.apache.poi.ss.usermodel.Font; +import org.apache.poi.ss.usermodel.HorizontalAlignment; +import org.apache.poi.ss.usermodel.IndexedColors; +import org.apache.poi.ss.usermodel.Row; +import org.apache.poi.ss.usermodel.Sheet; +import org.apache.poi.ss.util.CellRangeAddress; +import org.apache.poi.ss.util.RegionUtil; +import org.apache.poi.xssf.usermodel.XSSFCellStyle; +import org.apache.poi.xssf.usermodel.XSSFColor; +import org.apache.poi.xssf.usermodel.XSSFWorkbook; + +import keml.Information; +import keml.Literal; +import keml.NewInformation; +import keml.PreKnowledge; + +/** + * A class to modify and adjust {@link XSSFWorkbook} to facilitate the construct + * of the analysis matrix (i.e., xlsx file) unique to the LAF version of keml. + *

This class basically follows the same structure the baseline version ({@link WorkbookController}) but with + * minor modification to accommodate for LAF-related analysis matrix.

+ */ +public class LAFWorkbookController { + + XSSFWorkbook wb; + Sheet sheet; + + XSSFCellStyle defaultStyle; + XSSFCellStyle headerStyle; + XSSFCellStyle headerMessageStyle; + XSSFCellStyle bigHeaderStyle; + XSSFCellStyle floatStyle; + XSSFCellStyle trustStyle; + XSSFCellStyle distrustStyle; + XSSFCellStyle neutTrustStyle; + XSSFCellStyle instructionStyle; + XSSFCellStyle factStyle; + XSSFCellStyle origLLMStyle; + XSSFCellStyle origOtherStyle; + + // data properties + int firstFreeColumn = 0; + private HashMap infoToRow; + + + /** + * Constructor for {@link LAFWorkbookController} + */ + public LAFWorkbookController() { + + infoToRow = new HashMap<>(); + + wb = new XSSFWorkbook(); + sheet = wb.createSheet("Trust"); + // we need two rows we might merge: + Row headers = sheet.createRow(0); + Row headers1 = sheet.createRow(1); + + Cell start = headers.createCell(0); + defaultStyle = (XSSFCellStyle) start.getCellStyle(); + defaultStyle.setAlignment(HorizontalAlignment.CENTER); + + //************* headers ***************** + Font headerFont = wb.createFont(); + headerFont.setBold(true); + headerStyle = wb.createCellStyle(); + headerStyle.setRotation((short)90); + headerStyle.setAlignment(HorizontalAlignment.CENTER); + //headerStyle.setFont(headerFont); + + headerMessageStyle = wb.createCellStyle(); + headerMessageStyle.setAlignment(HorizontalAlignment.LEFT); + //headerMessageStyle.setFont(headerFont); + + bigHeaderStyle = wb.createCellStyle(); + bigHeaderStyle.setAlignment(HorizontalAlignment.CENTER); + Font bigHeaderFont = wb.createFont(); + //bigHeaderFont.setBold(true); + bigHeaderFont.setFontHeight((short) 360); + bigHeaderStyle.setFont(bigHeaderFont); + + start.setCellValue("Time"); + start.setCellStyle(headerStyle); + headers1.createCell(0); + sheet.addMergedRegion(new CellRangeAddress(0, 1, 0, 0)); + + + Cell i = headers.createCell(1); + i.setCellValue("Message"); + i.setCellStyle(headerMessageStyle); + headers1.createCell(1); + sheet.addMergedRegion(new CellRangeAddress(0, 1, 1, 1)); + + // for count of arguments for a given claim + i = headers.createCell(2); + i.setCellValue("#Arg+"); + i.setCellStyle(headerStyle); + headers1.createCell(2); + sheet.addMergedRegion(new CellRangeAddress(0, 1, 2, 2)); + + // for count of arguments against a given claim + i = headers.createCell(3); + i.setCellValue("#Arg-"); + i.setCellStyle(headerStyle); + headers1.createCell(2); + sheet.addMergedRegion(new CellRangeAddress(0, 1, 3, 3)); + + firstFreeColumn=4; + + // *********** styles ******************* + + CellStyle floatStyle = wb.createCellStyle(); + floatStyle.setDataFormat(wb.createDataFormat().getFormat("0.0#")); + + // additional color styles: + // *************** Trust **************** + trustStyle = wb.createCellStyle(); + trustStyle.setDataFormat(floatStyle.getDataFormat()); + trustStyle.setAlignment(HorizontalAlignment.CENTER); + trustStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#339966"), null)); + trustStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + // *************** Distrust ************* + distrustStyle = wb.createCellStyle(); + trustStyle.setDataFormat(floatStyle.getDataFormat()); + distrustStyle.setAlignment(HorizontalAlignment.CENTER); + distrustStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#FF5F5F"), null)); + distrustStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + // *************** neutral about trust ************* + neutTrustStyle = wb.createCellStyle(); + trustStyle.setDataFormat(floatStyle.getDataFormat()); + neutTrustStyle.setAlignment(HorizontalAlignment.CENTER); + neutTrustStyle.setFillForegroundColor(IndexedColors.LIGHT_YELLOW.getIndex()); + neutTrustStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + // ************** isFact ************* + factStyle = wb.createCellStyle(); + factStyle.setAlignment(HorizontalAlignment.CENTER); + factStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#99CC00"), null)); + factStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + + // ************* is Instruction ********** + instructionStyle = wb.createCellStyle(); + instructionStyle.setAlignment(HorizontalAlignment.CENTER); + instructionStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#FFCC00"), null)); + instructionStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + + // *************** origin LLM style************** + origLLMStyle = wb.createCellStyle(); + origLLMStyle.setAlignment(HorizontalAlignment.LEFT); + origLLMStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#CCFFFF"), null)); + origLLMStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + + // *************** origin Other style************** + origOtherStyle = wb.createCellStyle(); + origOtherStyle.setAlignment(HorizontalAlignment.LEFT); + origOtherStyle.setFillForegroundColor(new XSSFColor(java.awt.Color.decode("#FFFF99"), null)); + origOtherStyle.setFillPattern(FillPatternType.SOLID_FOREGROUND); + + } + + + public void initializeForLAF(List newInfos, List preKnowledge, Map literals2String, Map> logicArgs) { + int offset=2; // adapted in loop + for (int i=0; i < preKnowledge.size(); i++) { + PreKnowledge pre = preKnowledge.get(i); + infoToRow.put(pre, offset); + Row r = sheet.createRow(offset++); + Cell t = r.createCell(0); + t.setCellValue(-1); + colorByIsInstruction(t, pre.isIsInstruction()); + Cell msg = r.createCell(1); + msg.setCellValue(literals2String.get(pre.getAsLiterals().getFirst()) + ": " + pre.getMessage()); + colorByOrigin(msg, false); + r.createCell(2).setCellValue(logicArgs.get(pre.getAsLiterals().get(0)).size()); + r.createCell(3).setCellValue(logicArgs.get(pre.getAsLiterals().get(1)).size()); + } + for (int i=0; i< newInfos.size();i++) { + NewInformation info = newInfos.get(i); + infoToRow.put(info, offset); + Row r = sheet.createRow(offset++); + Cell t = r.createCell(0); + t.setCellValue(info.getTiming()); + colorByIsInstruction(t, info.isIsInstruction()); + Cell msg = r.createCell(1); + msg.setCellValue(literals2String.get(info.getAsLiterals().getFirst()) + ": " + info.getMessage()); + colorByOrigin(msg, info.getSourceConversationPartner().getName().equals("LLM")); + r.createCell(2).setCellValue(logicArgs.get(info.getAsLiterals().get(0)).size()); + r.createCell(3).setCellValue(logicArgs.get(info.getAsLiterals().get(1)).size()); + } + } + + public void addTrustsForLAF(Map> hCatPlus, Map> hCatMinus, HashMap trusts, String name) { + Row headers0 = sheet.getRow(0); + Cell i = headers0.createCell(firstFreeColumn); + i.setCellValue(name); + i.setCellStyle(bigHeaderStyle); + sheet.autoSizeColumn(firstFreeColumn); + i = headers0.createCell(firstFreeColumn+1); + sheet.addMergedRegion(new CellRangeAddress(0, 0, firstFreeColumn, firstFreeColumn+2)); + + Row headers = sheet.getRow(1); + i = headers.createCell(firstFreeColumn); + i.setCellValue("hCat+"); + i.setCellStyle(headerMessageStyle); + + i = headers.createCell(firstFreeColumn+1); + i.setCellValue("hCat-"); + i.setCellStyle(headerMessageStyle); + + i = headers.createCell(firstFreeColumn+2); + i.setCellValue("logAccumulator"); + i.setCellStyle(headerMessageStyle); + + + trusts.forEach((info, score) -> { + int rowIndex = infoToRow.get(info); + Row current = sheet.getRow(rowIndex); + current.createCell(firstFreeColumn).setCellValue(hCatPlus.get(info).toString()); + current.createCell(firstFreeColumn+1).setCellValue(hCatMinus.get(info).toString()); + setAndColorByValue(current.createCell(firstFreeColumn+2), score); + }); + setBorderLeft(firstFreeColumn); + + firstFreeColumn += 3; + } + + private void sizeColumns() { + for (int i = 0; i < firstFreeColumn; i++) { + sheet.autoSizeColumn(i); + } + } + + private void setBorderLeft(int columnIndex) { + CellRangeAddress hdr = new CellRangeAddress(0, 2, columnIndex, columnIndex); + RegionUtil.setBorderLeft(BorderStyle.MEDIUM, hdr, sheet); + RegionUtil.setLeftBorderColor(IndexedColors.BLACK.getIndex(), hdr, sheet); + //use messageMap to determine row count + CellRangeAddress adr = new CellRangeAddress(2, infoToRow.size()+1, columnIndex, columnIndex); + RegionUtil.setBorderLeft(BorderStyle.MEDIUM, adr, sheet); + RegionUtil.setLeftBorderColor(IndexedColors.WHITE.getIndex(), adr, sheet); + } + + private void colorByIsInstruction(Cell cell, boolean isInstruction) { + if (isInstruction) { + cell.setCellStyle(instructionStyle); + } else { + cell.setCellStyle(factStyle); + } + } + + private void colorByOrigin(Cell cell, boolean isLLM) { + if (isLLM) { + cell.setCellStyle(origLLMStyle); + } else { + cell.setCellStyle(origOtherStyle); + } + } + + private void setAndColorByValue(Cell cell, float value) { + cell.setCellValue(value); + if (value > 0.0f) + cell.setCellStyle(trustStyle); + else if (value <0.0f) + cell.setCellStyle(distrustStyle); + else + cell.setCellStyle(neutTrustStyle); + } + + public void write(String file) throws IOException { + sizeColumns(); + String path = FilenameUtils.removeExtension(file) + "-trust.xlsx"; + try(FileOutputStream o = new FileOutputStream(path)) { + wb.write(o); + wb.close(); + } + } + +} diff --git a/src/keml/analysis/LogicArgument.java b/src/keml/analysis/LogicArgument.java new file mode 100644 index 0000000..3688259 --- /dev/null +++ b/src/keml/analysis/LogicArgument.java @@ -0,0 +1,121 @@ +package keml.analysis; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; + +import keml.Junction; +import keml.Literal; +import keml.LogicExpression; + +/** + * Object representation of a logical argument + */ +public class LogicArgument { + + /** {@link Literal} claim of this argument */ + Literal claim; + /** {@link LogicExpression} premises of this argument*/ + List premises; + + /** + * Constructor for {@link LogicArgument} + * @param claim {@link Literal} claim of the to-be-created instance + */ + public LogicArgument(Literal claim) { + this.claim = claim; + this.premises = new ArrayList<>(); + } + + /** + * adds a premise to the list of {@link LogicArgument#premises} for this instance. + * @param le {@link LogicExpression} + */ + public void addPremise(LogicExpression le) { + premises.add(le); + } + + /** + * getter for this instance's {@link LogicArgument#claim}. + * @return claim {@link Literal} + */ + public Literal getClaim() { + return claim; + } + + /** + * getter for this instance's {@link LogicArgument#premises}. + * @return premises as List of {@link LogicExpression}s + */ + public List getPremises() { + return premises; + } + + + /** + * Utility static method that creates a String representation of a {@link Junction} using + * recursion and a map of {@link Literal}s to their unique String symbol + * @param j target junction + * @param content String content so far (facilitates recursion) + * @param literals2String Map of literals to their String symbol + * @return String representation of the target junction + */ + private static String junctionAsString(Junction j, String content, Map literals2String) { + + for (int i = 0; i < j.getContent().size(); i++) { + LogicExpression le = j.getContent().get(i); + if (le instanceof Literal) { + content += literals2String.get((Literal) le); + } else { // if junction contains junctions + content += junctionAsString((Junction) le, content, literals2String); + } // conjunct or disjunct the content so far + if (i < j.getContent().size() - 1) + content += j.isDisjunction() ? " || " : " && "; + + } + + return content + ")"; + } + + + /** + * Utility static method that creates a String representation of the premises of a {@link LogicArgument}. + * @param la target logic argument + * @param literals2String Map of {@link Literal}s to their String symbol + * @return String representation of the target argument's premises + */ + private static String premisesAsString(LogicArgument la, Map literals2String) { + String premises = ""; + for (int i = 0; i < la.getPremises().size(); i++) { + LogicExpression le = la.getPremises().get(i); + if (le instanceof Literal) { + premises += literals2String.get((Literal) le); + if (!((Literal) le).equals(la.getClaim())) + premises += " => " + literals2String.get(la.getClaim()); + + } else { //if premises contain junctions + premises += junctionAsString((Junction) le, "(", literals2String) + " => " + literals2String.get(la.getClaim()); + } + // separate premises so far + if (i < la.getPremises().size() - 1) { + premises += ", "; + } + + } + + return premises; + } + + + /** + * Utility static method that creates a String representation of a {@link LogicArgument}. + * @param la target argument + * @param literals2String Map of {@link Literal}s to their String symbol + * @return String representation of the target argument + */ + public static String asString(LogicArgument la, Map literals2String) { + return "<{" + premisesAsString(la, literals2String) + "}, " + literals2String.get(la.getClaim()) + ">"; + } + + +} diff --git a/src/keml/analysis/LogicUtilities.java b/src/keml/analysis/LogicUtilities.java new file mode 100644 index 0000000..fbe0a36 --- /dev/null +++ b/src/keml/analysis/LogicUtilities.java @@ -0,0 +1,166 @@ +package keml.analysis; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import keml.Information; +import keml.Junction; +import keml.Literal; +import keml.LogicExpression; +import keml.PreKnowledge; +/** + * Utility class containing a myriad of static methods that help the analysis of LAF keml models + */ +public class LogicUtilities { + + /** + * static method that creates a map of rebuttals of each claim {@link Literal} for a list of {@link LogicArgument}s + * @param logicArguments target map of literals and their list of arguments + * @return Map of literals and logic arguments that rebut them + */ + public static Map> createRebuttalsMap(Map> logicArguments) { + Map> rebuttals = new HashMap<>(); + for (Literal l : logicArguments.keySet()) { + Literal counterClaim = findCounterLiteral(l); //negated the claim + rebuttals.put(l, logicArguments.get(counterClaim)); //find the arguments for the negated claim + } + return rebuttals; + } + + + /** + * static method that creates a map of undercuts of each {@link LogicArgument} for a list of arguments + * @param logicArguments target map of {@link Literal}s and their list of arguments + * @return Map of logic arguments and their {@link ArgumentTree}, where children undercut their parent node + */ + public static Map createUndercutTreesMap(Map> logicArguments) { + Map undercutTrees = new HashMap<>(); + //create empty trees for each argument of each literal + for (List args : logicArguments.values()) { + for (LogicArgument arg : args) { + ArgumentTree at = new ArgumentTree(arg); + undercutTrees.put(arg, at); + } + } + // connect the created trees whenever there exists an undercut relation between them + for (LogicArgument arg : undercutTrees.keySet()) + undercutTrees = findUndercuts(arg, undercutTrees, logicArguments); + + return undercutTrees; + } + + + /** + * static method that finds all undercuts for a given {@link LogicArgument} by connecting undercut trees + * @param la target argument + * @param undercutTrees Map of logic arguments and their corresponding {@link ArgumentTree} + * @param logicArguments Map of {@link Literal} and their logic arguments + * @return undercutTrees after connecting trees that have undercut relations + */ + private static Map findUndercuts(LogicArgument la, Map undercutTrees, Map> logicArguments) { + ArgumentTree at = undercutTrees.get(la); // root node + List premises = la.getPremises(); // premises of root + + Literal counterLiteral; //negated root + + for (LogicExpression premise : premises) { + if (premise instanceof Literal) { + counterLiteral = findCounterLiteral((Literal) premise); + if (logicArguments.containsKey(counterLiteral)) {// if counter literal has argument(s) then it undercuts the root argument! + for (LogicArgument undercutter : logicArguments.get(counterLiteral)) { + if (!undercutter.getClaim().equals(la.getClaim())) + at.addChild(undercutTrees.get(undercutter)); + } + } + + } else { // if premises contain junctions + for (LogicExpression junctionContent : ((Junction) premise).getContent()) { + // TODO this currently doesn't handle nested junctions + counterLiteral = findCounterLiteral((Literal) junctionContent); + if (logicArguments.containsKey(counterLiteral)) // if counter literal has argument(s) then it undercuts the root argument! + for (LogicArgument undercutter : logicArguments.get(counterLiteral)) { + at.addChild(undercutTrees.get(undercutter)); + } + } + } + } + return undercutTrees; + } + + + /** + * static method that finds the negated version of a given {@link Literal} + * @param l target literal + * @return negated version of target literal + */ + private static Literal findCounterLiteral(Literal l) { + Information sourceInfo = l.getSource(); + + if (l.isNegated()) + return sourceInfo.getAsLiterals().get(0); //non-negated are stored first during parsing + else + return sourceInfo.getAsLiterals().get(1); //negated are stored last during parsing + } + + + /** + * static method that computes the hCategorizer value for a given {@link ArgumentTree}. + * @param at target argument tree + * @return float categorisation value + */ + public static float hCategorizer(ArgumentTree at) { + + List incomingEdges = at.getChildren(); + if (incomingEdges.isEmpty()) //if no children + return 1.0f; + + float sum = 0.0f; + for (ArgumentTree edge : incomingEdges) { + sum += hCategorizer(edge); // recursively find hCat value for children + } + + return 1.0f / (1.0f + sum); + } + + + /** + * static method that computes the hCategorizer value for a flat tree. + *

this method is mainly used to compute values of rebuttal trees

+ * @param numberOfChildren int number of children the root of this flat tree has + * @return float categorisation value + */ + public static float flatTreeHCategorizer(int numberOfChildren) { + + return 1.0f/(1.0f + (float) numberOfChildren); + } + + + /** + * b>static method that computes the logAccumulator value given a list of hCategorisations for + * a given argument and one against the argument. + * @param forCategorizations List of categorisation values for a given argument + * @param againstCategorizations List of categorisation values against a given argument + * @return float log-accumulated value + */ + public static float logAccumulator(List forCategorizations, ListagainstCategorizations) { + // if the claim has no arguments for it or against + if (forCategorizations.isEmpty() && againstCategorizations.isEmpty()) + return 0.0f; + + float forSum = 1.0f; + float againstSum = 1.0f; + + for (float value : forCategorizations) { + forSum += Math.abs(value); + } + + for (float value : againstCategorizations) { + againstSum += Math.abs(value); + } + + double logResult = Math.log(forSum) - Math.log(againstSum); + + return (float) logResult; + } +}