Questions that cannot be traced to the material are the ones students complain about
Writing assessment questions is slow, not because inventing the questions takes time, but because you have to ensure each question can be answered from what was actually taught and sits at a difficulty level that matches the students' current ability.
Questions generated by a model easily go beyond the scope of the material. A model that is familiar with the subject will readily ask about something that was never mentioned in the lectures, and this only becomes apparent when the exam takes place.
The second issue is difficulty. When a mix is requested, generation yields mostly recall questions, because those are the easiest to prepare. An exam of recall questions tests memory rather than understanding.
Traceability addresses both issues. Each item includes the source it was drawn from and its stated difficulty level, so the teacher looking at the collection can see at a glance how well the material is covered and how it is balanced, rather than having to read 40 questions.
How the ActionFlow runs on the canvas
Course material enters through a Document Upload node. The PDF4me node then extracts the slides or reading materials provided in PDF format.
The Extract Data node separates the material by topic, which is why you can check coverage rather than assume it.
The Loop node goes through the topics. Inside the loop, a Generate Object node creates items for each topic according to a difficulty distribution that you define: recall, application, and analysis. For each item it returns both the source passage and the level.
The design choice is to generate topic by topic rather than across all the material, which keeps coverage fixed. If the system runs a single generation covering all the material, it ends up focusing on whatever it found most interesting.
The Math node tallies items by topic and level. The Filter node identifies any underrepresented topics, so the educator can see the gap before the set is used.
A Validate node rejects any item whose source passage does not contain the answer. This check picks up cases of drift outside the material.
A Merge node puts the set together. A Create Excel node creates the question bank with its metadata. A Notion node stores the study cards.
The Human in the Loop node sends all the material to the educator. Nothing is sent to students through the flow.
Nodes this ActionFlow uses
- Document Upload (Data) plus PDF4me (Data): extract the course material.
- Extract Data (Data): splits by topic so coverage can be measured.
- Loop (Control): generates per topic rather than across everything at once.
- Generate Object (AI Core): returns items with source passage and difficulty level attached.
- Validate (Data): rejects items whose source passage does not contain the answer.
- Math (Util) plus Filter (Data): count by topic and level, and flag under-represented topics.
- Create Excel (Data): produces the question bank with its metadata.
- Slack (Human in the Loop): routes the set to the educator. Nothing reaches students automatically.
Frequently asked questions
Start building AI workflows
Create a free account, open a template or a blank canvas, and run your first ActionFlow.
Newsletter
Get product updates
New nodes, agents, and product notes. We send mail only when we have something worth opening.