CT-GenAI sample questions

Question 1

A financial institution is implementing a Generative AI solution to assist in creating test cases for a legacy mainframe system. The system logs are verbose, often exceeding 50,000 words per transaction cycle. The team attempts to paste these logs directly into a standard LLM to identify error patterns, but the model returns incomplete analysis or errors regarding input length. Which fundamental constraint of Large Language Models is the team encountering, and what is the most architecturaly sound resolution?

Show answer & explanation

Correct answer: B

The context window limits the amount of text (tokens) the model can process in a single interaction. Mainframe logs often exceed standard limits. Chunking the data or using extended context models is the correct architectural solution.

Accepted Answer

The input exceeds the model's Context Window; the team should implement a chunking strategy or utilize a model with a significantly larger context window (e.g., 128k+ tokens).

Answer

The model is suffering from hallucination due to the complexity of mainframe logs; the team should switch to a higher temperature setting to increase creativity.

Answer

The model lacks specific training on mainframe terminology; the team must fine-tune the model before it can process any logs.

Answer

The logs contain binary data that causes tokenization failures; the team must manually transcribe the logs into natural language summaries.

Question 2

Which of the following scenarios best illustrates the application of a Multimodal Large Language Model (MLLM) in a software testing context?

Accepted Answer

A tester inputs a screenshot of a user interface along with the requirements text, asking the AI to identify discrepancies between the visual implementation and the specification.

Answer

A tester provides a Python script and asks the AI to translate it into Java code for a new test framework.

Answer

A tester inputs a large CSV file of test data and asks the AI to generate a summary of the data distribution.

Answer

A tester chains multiple prompts together to break down a complex end-to-end test scenario into individual steps.

Question 3

In the context of Generative AI for software testing, how does 'Tokenization' fundamentally impact the cost and processing of test artifacts?

Accepted Answer

Tokenization breaks input text into smaller units (sub-words/characters) which serves as the unit of measurement for both the model's processing limits and billing.

Answer

Tokenization encrypts the test data to ensure security during processing, increasing processing time but reducing data leakage risks.

Answer

Tokenization converts natural language prompts into executable SQL queries for database testing.

Answer

Tokenization is the process of assigning unique IDs to test cases to track them through the Generative AI lifecycle.

Question 4

A test manager is explaining the difference between Discriminative AI and Generative AI to stakeholders. Which comparison accurately reflects their roles in software testing?

Accepted Answer

Discriminative AI is used for identifying whether a test passed or failed based on boundaries; Generative AI creates new test cases or code based on learned patterns.

Answer

Discriminative AI creates new test data based on patterns; Generative AI classifies existing defects as high or low priority.

Answer

Discriminative AI requires a prompt engineer; Generative AI works autonomously without human input.

Answer

There is no difference; both terms refer to Large Language Models used in testing.

Question 5

When using an LLM to generate unit tests, a developer notices the output varies significantly each time the same prompt is submitted. Which parameter should be adjusted to make the output more deterministic for regression testing purposes?

Accepted Answer

Decrease the 'Temperature' value (approaching 0).

Answer

Increase the 'Top-K' value to limit the vocabulary usage.

Answer

Increase the 'Temperature' value (approaching 1 or higher).

Answer

Enable 'Frequency Penalty' to force repetition.

Question 6

Which of the following best describes 'Instruction-Tuned' models compared to 'Foundation' models in the context of software testing?

Accepted Answer

Instruction-Tuned models have been fine-tuned on datasets of instructions and responses, making them better at following specific testing commands than raw Foundation models.

Answer

Foundation models are optimized for following specific tasks like 'Generate a test case', while Instruction-Tuned models are just raw data predictors.

Answer

Instruction-Tuned models are smaller and run locally, whereas Foundation models must always run in the cloud.

Answer

There is no functional difference; the terms are marketing synonyms.

Question 7

A test analyst is crafting a prompt to generate Gherkin scenarios. They explicitly state: "You are a Senior QA Automation Engineer with 10 years of experience in E-commerce." Which component of a structured prompt does this sentence represent?

Accepted Answer

Role/Persona

Answer

Context

Answer

Instruction

Answer

Constraint

Question 8

You are tasked with using an LLM to generate test cases for a login feature. You initially send a prompt: "Write test cases for login." The output is generic and unusable. You then refine the prompt to include the specific requirements: "The login accepts email and password. Password must be 8 chars. Lockout after 3 attempts." Which prompt engineering principle are you applying to improve the result?

Accepted Answer

Adding Context

Answer

Zero-shot Prompting

Answer

Prompt Chaining

Answer

Role prompting

Question 9

A tester wants to generate equivalence partition test data for a purely mathematical function. They provide the LLM with the function logic and then ask for the test data immediately, but the model makes calculation errors. Which prompting technique involves asking the model to "Think step-by-step" to improve logical accuracy?

Accepted Answer

Chain-of-Thought (CoT) Prompting

Answer

Few-Shot Prompting

Answer

Zero-Shot Prompting

Answer

Meta Prompting

Question 10

You need to generate test cases that strictly follow a specific JSON format for your test management tool. You provide three complete examples of the input requirement and the corresponding JSON output in your prompt. What technique are you utilizing?

Accepted Answer

Few-Shot Prompting

Answer

Zero-Shot Prompting

Answer

Chain-of-Thought Prompting

Answer

Self-Consistency Prompting

ISTQB Certified Tester - Testing with Generative AI Free Sample Questions

Lifetime One

Two

Pro Four

Premium Twelve

CT-GenAI Sample Questions

Question 1

Question 2

Question 3

Question 4

Question 5

Question 6

Question 7

Question 8

Question 9

Question 10

Trusted by professionals at