Proj 4a: Fitch Segment Data

Our task in this project is to leverage AI to help find business activity data embedded within large pdf files, which may be hundreds of pages long.

This is one of the first (and most difficult) steps taken by Sustainable Fitch, a company that provides entity ratings based on Environmental, Social, and Governance (ESG) characteristics. This is information of interest to discriminating large investors. Sustainable Fitch is a subsidiary of Fitch, one of the big three ratings companies that evaluates large corporations, banks, and even governments (e.g. the US Government in '23). Read the Sustainable Fitch overview document (also embedded below) that describes the project goals, and review a Fitch list of business activities that is similar to what we will be doing.

Details

Keep track of our work so you can make the case for your solution providing value to Fitch. We want to keep track of how long it takes to manually process each pdf file, documenting a repeatable process that can be handed off to others, describing the optimal mix of human effort and use of AI tools.

Accessing the Documents

This google drive folder has the 100 company documents provided by Fitch. They are grouped into five zip files with 20 company documents each, for your convenience. Also in that folder is the sample_ground_truth spreadsheet with ground truth information for four entities.

Searching the Documents

Start with this spreadsheet (pictured at right).

In the Fitch List tab, find your group number and then your name to see the four company documents assigned to you, as pictured at right.
The README tab has links to the Activity Report form you will need to use after finding the activity data. For each of your companies, in the Activity Report form, you will be prompted to fill in values for each of the following:
1. Company Number: The company's four or five digit number (e.g. see image at right)
2. Company Name: The first one or two words of the company's name (e.g. see image at right)
3. Whether or not you found the business activities within the large pdf
4. The page number where you found the values
5. Time Period: Fiscal year or reporting period
6. Currency (e.g. USD, EUR, etc.)
7. Scale (millions, billions, etc.)

Then for each business activity you will use the same form to fill in the following values:

Company Number: The company's four or five digit number (e.g. see image at right)
Company Name: The first one or two words of the company's name (e.g. see image at right)
Company Label: The company's label for this business activity
Amount: The numerical amount
NACE Segment Name: The NACE segment name for the business activity. See below on help for this.

How to Select NACE Segments

Once you have identified the business activities, for each one select the best-matching NACE segment. To help you, use the NACE_Rev2 spreadsheet (using the Outline tab, shown at right), or explore using the interactive content on this page below.

The schedule through the end of the semester is roughly as shown below. See the course website for the latest schedule information.

Submission & Evaluation

Place the components shown below on your website. Each of these will be graded 0..3 as follows, for a total possible maximum of 12 points:
0 points: Not done

1 point: Partially done, with significant parts incomplete

2 points: Almost everything is there
3 points: Done completely, with all aspects addressed. Includes appropriate visual elements.

Website Required Components:

Link or embedding of your group's planning document, indicating
1. What exploratory (learning) you did or still need to do outside of class
2. Timeline - Who/What/When – work-back from deadline, what needs to happen when, by whom
Link or embedding of your group's presentation slides. This should include how your solution provides value, such as by reducing risk, improving efficiency, increasing trust, or enabling scale.
Your personal contributions to the project as a STAR story (Situation, Task, Action, Result), the goal being that someday, when you are interviewing for a job, you could bring this up and use it to tell a compelling narrative about this project.
A Description of your solution. This should include:
1. Extraction Solution
  1. Documented workflow (step-by-step process description)
  2. Code, prompts, or tool configurations used
  3. Instructions for running the solution on new documents, including links
2. Extracted Data File
  1. Excel or CSV file with all extracted segments in structured format
  2. Columns should include: Company Name, Document Name, Segment Name, Revenue Value, Revenue Unit, Time Period, Page/Section Reference
3. Performance Analysis
  1. Summary of extraction success rate (how many segments captured vs. expected)
  2. Analysis of failure modes (what types of cases does your solution miss?)
  3. Reconciliation results (do extracted segments sum to reported totals?)
4. Reflection Document
  1. What was straightforward to extract with AI?
  2. What required the most iteration and refinement?
  3. What types of segment disclosures are most challenging?
  4. Where would human review still be necessary?
  5. How confident are you in the accuracy and completeness of your extractions?

See the Sustainable Fitch official project description, also embedded below. It includes descriptions of some possible extensions to the project. See also the Fitch business activities sample, containing similar content to what we are doing, though with different segment labels.

Extracting Segment Data

Page updated

Report abuse