AI-Classifier: Automating document relevance sorting in workfiles

Modified on Fri, 31 Oct at 4:12 PM

TABLE OF CONTENTS

What is AI-Classifier?
Using AI-Classifier: A two-step process
- Step 1: Initial learning phase
- Step 2: Iterative Improvement
Frequent questions and answers
- Can I export this score like any other data from a workfile where the AI-Classifier is activated?
- Will my colleagues be able to see the score for each document and have the ability to change it?
Recommended articles to read

What is AI-Classifier?

AI-Classifier is a feature that automatically sorts patents within a Workfile into Relevant and Not-Relevant categories. It relies on Machine Learning algorithms and begins with a learning phase, during which the AI-Classifier creates a classification rule based on user's manually classified documents.

Explanation with an image : the AI-classifier will draw a "facial composite" of the perfect patent family, based on manually marked as relevant documents. Then the classifier will compare each family to this facial composite and build up a score of similarity.

By going beyond traditional filters, AI-Classifier captures all potentially relevant data. This improves key document detection and reduces the risk of missed insights, based on the accuracy achieved during the learning phase.
It can thus speed up your information sharing and decision-making processes. This feature is included at the Premium level.

Using AI-Classifier: A two-step process

Step 1: Initial learning phase

Begin by manually classifying a sample of documents. This step teaches the AI your criteria for what is considered Relevant or Non-Relevant.

Activate AI-Classifier
- Open a new or existing Workfile.
- Click “Enable Auto-Classifier” at the top-right corner of the hitlist. A pop-up window will appear, simply click “OK” to activate the feature.
- Once enabled, a classification label will be added to each document title in the list.
Start the Learning Phase
- Manually classify at least 10 documents, including both Relevant and Not-Relevant items.
  For optimal results, make sure to include at least 2 Relevant documents.
- To classify a document, click on its classification label and select either Relevant or Not-Relevant.
  If you're unsure about a document, you can leave it unclassified, the AI will only learn from the ones you label.
- Please note that:
  - The more documents you classify manually, the better the AI will perform
  - Staying within a specific subject or theme improves classification accuracy : tagging 200 families as relevant, could not always produce better results than tagging 30 families to start with. As often these 30 families are actually close/similar to each other, where the 200 could be scattered among slightly different subjects.
Run Auto-Classification
- Once enough documents are classified, open the "AI-Classifier monitor" menu.
  A pop-up window will display the number of Relevant and Not-Relevant documents you've labelled.
- Click “Run AI-Classifier” to automatically sort the unclassified documents based on your input.
- You can disable this feature at any time from the same menu if needed.

Step 2: Iterative Improvement

After the first run of the AI-Classifier, the algorithm calculates a threshold value based on the initial learning phase.

Documents with a rating above this threshold are considered Relevant.
Documents below the threshold are considered Not-Relevant.

To improve classification accuracy, it is recommended to review the automatically ranked documents and reiterate the process. Each iteration helps the AI refine its understanding and better align with your relevance criteria.

Note: if this threshold is drastically decreasing after the second run of AI-classifier, you have marked as Relevant, one or several documents which are far/distant of the original "facial composite" (see in Introduction section).

Refinement Workflow

Review Automatically Ranked Documents
- Examine the documents and manually reclassify them as Relevant or Not-Relevant based on your assessment.
- Documents with a classification label score between 1% and 99% are automatically classified, while manually designated documents display a score of 100% (Relevant) or 0% (Not-Relevant).
- Pay special attention to those with scores close to the threshold value, as they are more likely to be borderline cases.
Re-run the AI-Classifier
- This allows the algorithm to refine its understanding based on your updated input.
- A new threshold value is calculated, and the documents are reassessed accordingly.
Repeat the Process
- Continue refining through successive iteration until the desired classification precision is achieved.

Once you’ve taught and refined the AI-Classifier it becomes more accurate, saving you time and and helping you find the right documents faster.

Note: The AI-Classifier adapts its learning per Workfile, meaning its behaviour may vary depending on the context.

A document marked as Not-Relevant in one Workfile may be considered Relevant in another, depending on the subject matter and classification criteria applied during the learning phase.

Frequent questions and answers

Can I export this score like any other data from a workfile where the AI-Classifier is activated?

Yes, you can export the classification score for each document. Please select the item "Metadata" from the available fields when configuring an export like :

Will my colleagues be able to see the score for each document and have the ability to change it?

Yes, then no. Once the AI-Classifier is activated in a workfile, any user of Orbit Intelligence will see the score and the value relevant/not relevant, as well as exporting this. This score and value could be considered as another custom field value , in read only.

However, only users with a proper licence AI-classifier (with the access level 'Write for the related workfile) may classify, and run the classifier.