Chat Parameters

'Temperature' determines how unpredictable or creative the model’s responses will be.

• Lower values (e.g., 0.2) produce more focused, deterministic answers, ideal for tasks requiring accuracy.
• Higher values (e.g., 0.8) increase randomness, making responses more varied and creative.

Configuration Guidance

Adjust 'Temperature' for creative tasks or when you want the model to explore more possibilities. For consistent, reliable answers, use lower values. It is recommended to modify either 'Temperature' or 'Top P', but not both.

For most data extraction and classification tasks, lowering the temperature to around 0.2 is recommended.

'Top P' (nucleus sampling) restricts the model to consider only the most probable tokens whose cumulative probability mass meets the specified threshold.

• Lower values (e.g., 0.1) make the output more focused by considering only the top 10% of likely tokens.
• Higher values (e.g., 1.0) allow for greater diversity by including more possible tokens.

Configuration Guidance

Use 'Top P' to fine-tune the diversity of responses. For narrow, precise answers, set a low value. For broader, more creative outputs, increase the value. Adjust either 'Top P' or 'Temperature', but not both.

'Presence Penalty' increases the likelihood that the model will discuss new topics by penalizing tokens that have already appeared in the response.

• Positive values (e.g., 1.0) strongly encourage new topics and discourage repetition.
• Negative values reduce this effect, allowing more repetition.

Configuration Guidance

Use 'Presence Penalty' to broaden the scope of generated content, especially when variety is desired. Combine with 'Frequency Penalty' for maximum reduction of redundancy.

'Frequency Penalty' discourages the model from repeating the same tokens, making responses less redundant.

• Positive values (e.g., 1.0) decrease repetition, resulting in more varied output.
• Negative values allow more repetition, which may be useful for certain tasks.

Configuration Guidance

Use 'Frequency Penalty' to minimize repeated phrases or lines, especially in summarization or creative writing. It can be combined with 'Presence Penalty' for broader topic coverage.