Skip to main content

Prompt-based experimentation FAQ

This article answers common questions about prompt-based experimentation (PBX).

Which LLM does PBX use as its thinking model?​

PBX uses GPT 5.2 high as its thinking model.

Which LLM does Kameleoon use for image generation?​

Kameleoon integrates with OpenAI gpt-image-1 to generate images using AI.

Can I use the images that Kameleoon AI generates?​

Yes, all images that Kameleoon AI generates are free for commercial use. Kameleoon automatically uploads them to your Image Library within the platform for easy access.

Can I edit the code that Kameleoon generates?​

Yes, you can edit code directly from the prompt-based interface. When you save the code, Kameleoon automatically generates a new version of your variant.

Can I use the Graphic editor on an AI-generated variant?​

No, you cannot use the Graphic editor on AI-generated variants. Although some edge cases might benefit from it, prompt-based experimentation handles all scenarios that the Graphic editor covers. To prevent conflicts, Kameleoon disables the visual tool for AI-generated experiments.

What types of variants can I create with prompts?​

You can create any variant that you manage with front-end code. However, Kameleoon cannot generate code for prompts that require backend logic or server-side changes. A developer familiar with your backend system must handle such cases.

Can I create multi-page experiments?​

Yes, you can prompt changes across multiple pages. Navigate to your chosen pages and apply prompts directly. Kameleoon automatically combines the code into a single variation.

However, test the complete experience using Simulation mode to ensure that the code on different pages does not conflict.

Can I prompt changes on elements that appear on hover?​

Yes, provided the page's DOM contains the HTML code for the hover-triggered element. If JavaScript injects the element dynamically after the hover interaction, Kameleoon’s AI might not detect or modify it accurately.

For best results, verify that:

  • The hover element exists in the initial HTML, even if hidden.
  • The structure is stable and does not generate asynchronously.

How can I test a variation across several product pages?​

After you create a variation on the first product page, navigate to the next product page. This action re-executes the AI-generated code.

Alternatively, use Simulation mode to test the experience across all relevant pages and ensure consistent behavior.

How does PBX differ from other vibe coding tools?​

Kameleoon designed PBX specifically for experimentation. It provides value through several key features:

  • Agentic architecture for experimentation: PBX handles complex experimentation logic, transforming your ideas into working variants directly on your website in minutes.
  • Browser-native integration: PBX embeds within Chrome and uses your site's full dynamic content, including the technology framework, design system, and the complete DOM state at a given time. This integration allows PBX to generate code that interacts with dynamic elements, such as React components, without breaking existing site functionality.
  • Visual contextual intelligence with human-in-the-loop (HITL): PBX ensures accuracy and control. If a prompt lacks context, PBX asks clarifying questions before it generates the variant.
  • Performance-optimized and accessible code: PBX minimizes performance impacts. It uses the native Kameleoon JavaScript API to manage flicker, performance issues, and dynamic components seamlessly.

Why does the same prompt generate different outputs when I run it multiple times?​

Our AI explores many possible ways to fulfill your request, so it does not produce a fixed result. The same prompt can yield different outputs each time.

This occurs because of:

  • Controlled randomness: The model varies its answers to avoid repetition and encourage creativity.
  • Prompt interpretation: Minor context differences can prompt the model to focus on different details.
  • Training diversity: The model draws on patterns from varied training datasets.

To get more consistent results, add constraints. Provide a mockup, design file, or define a Master Prompt that sets overarching rules and brand context for all your generations.

Does prompt-based experimentation work on all websites?​

You can use PBX on most websites, including single-page applications. Each time you submit a prompt, Kameleoon sends page context, such as HTML code or screenshots, to help the AI interpret the content. However, the AI might not properly handle context within iframes or a shadow DOM. If you frequently experience this issue, contact your Customer Success Manager.

Can PBX handle complex experiments?​

PBX excels at tasks such as:

  • Text or content changes, including headlines, CTAs, and disclaimers.
  • Style updates, like colors, fonts, and layout tweaks.
  • Banners and content insertions.
  • Simple interactive changes, including button repositioning, links, and modals.

You can also use PBX for multi-step flows, galleries, or custom interactive elements. For advanced cases, use a hybrid approach. First, let PBX generate the initial implementation, then refine it manually. This method combines the speed of automation with the precision of custom development.

How reliable is the code that PBX generates?​

PBX generates clean, secure, responsive, and accessible code. It often creates more robust solutions than manual coding, particularly for modern websites and single-page applications (SPAs). PBX follows QA best practices, such as cross-browser testing and visual checks.

The generated code also respects your project configurations:

  • PBX applies SPA settings and custom attributes automatically.
  • It honors exclusion rules.

These considerations ensure that generated code integrates smoothly with your setup and adheres to the rules for manually coded experiments.

Can PBX generate a variant by retrieving content from a different page?​

No. PBX relies on the context of the page you are viewing. It cannot browse other URLs to retrieve content or code.

However, if you provide an endpoint or web service in your prompt, PBX can generate a variant to load and display that data. For instance, if you want an urgency tooltip showing recent purchases and you supply the endpoint, PBX can generate the necessary code.

PBX cannot automatically fetch logic from another page. For example, it cannot fetch the "Add to Cart" logic from a product page to apply it to a listing page. However, if you instruct it to "call this endpoint to add the product to the cart," PBX might generate the correct variant.

If your listing page includes a quick-view overlay that contains the logic, apply your prompt while the overlay is open. The available code context increases the chance of creating a functional variant.

Can I use a design mockup to create a variant?​

Yes, you can use a design or mockup. Click the + icon > Add an image to upload your file. However, a mockup alone is usually insufficient. For best results, use your design alongside clear, detailed instructions.

Treat PBX like a developer: provide specific details for the best output.

Example of a good prompt with a mockup:

  • Attach your mockup or design file.
  • Describe the elements to change.
  • Specify dimensions, colors, and spacing.
  • Explain interactive behaviors.
  • Note technical requirements.

How do SPA settings affect PBX?​

PBX adopts your project's single-page application (SPA) settings. If you configured these settings (see Set up an experiment on a single-page app), PBX automatically uses:

  • Your custom attributes.
  • Your exclusion rules.
  • Any other SPA-specific configurations.

Can I submit follow-up prompts to fix incorrect output?​

Yes, you can refine the output with follow-up prompts, but keep prompt history in mind.

What PBX remembers:

  • The code it most recently generated (used as context).

What PBX doesn't remember:

  • Your previous prompt instructions.
  • The full conversation history.
  • Mockups or sketches added in earlier prompts.

Why this matters: Vague follow-up prompts like "Fix the issue with X and Y" might fail because PBX lacks full context.

Best practices for follow-up prompts:

  • Less effective: "The arrow navigation doesn't work."
  • More effective: "In the new carousel, the arrow navigation does not work as expected. Fix it using the code generated from my previous prompt. The arrows should appear on the left and right and advance one image per click."

Be specific in follow-up prompts:

  • Identify the element or feature to fix.
  • Describe the expected behavior.
  • Reference the generated code explicitly.
  • Include relevant technical details.

Can I upload a file to use as an asset in my prompt-based experiment?​

Yes. Click + > Add an image to upload your file.

What is the difference between an asset and a mockup when I upload an image?​

When you upload or paste an image, PBX asks you to choose how to use it:

  • Asset: PBX inserts the image file directly into your variation (for example, as a banner, background, or logo).
  • Mockup: PBX uses the image as a visual reference to recreate the design using code, rather than inserting the image itself.

How does PBX protect personal or sensitive data?​

PBX keeps your data secure and meets industry standards, including those for regulated sectors.

PBX invokes the LLM only when you create an experiment from a webpage you control. This setup ensures that:

  • PBX sends no end-user data to the LLM.
  • The model processes only the page content visible to you as the creator, excluding site visitor data.
  • PBX does not dynamically generate experiments for each user; it creates them once from your environment.

PBX will not access or transmit personal or confidential information unless you include it in the page or prompt. Avoid including such data.

note

Kameleoon continuously improves the PBX workflow to maintain high privacy and compliance standards.

Can I work on other variants while PBX generates a variant?​

Yes. You can build multiple variants in parallel. For more information, see Work on multiple variants in parallel.

Why do I see a white screen in the PBX editor?​

This issue occurs on secure pages, such as logged-in areas, that block loading within an iframe. The PBX editor uses an iframe to safely edit webpages. Your site's security settings might block the iframe content, resulting in a white screen.

How to fix it:

Ask your development team to modify the "frame-busting" logic. The code must check if the frame's name is kameleoon-pbx-preview. If it matches, the logic should allow the page to load.

Example code:

if (window.self !== window.top && window.frameElement?.name !== 'kameleoon-pbx-preview') {Your existing redirect/blocking logic}

Can I prompt PBX to copy a feature from another website?​

No. PBX relies on the context of the page you are viewing. You cannot instruct PBX to copy a feature from an external URL because it does not browse other pages to retrieve content.

Instead, describe the feature you want. For example, instead of saying "Copy the sliding product carousel from Amazon," say "Create a horizontal product carousel with left and right navigation arrows, displaying four products at a time with images, titles, and prices."

For more details on effective prompts, view How to write effective prompts.

Why can't I log in to PBX?​

If you cannot log in, your browser likely blocks third-party cookies. PBX requires a session cookie from the Kameleoon domain for authentication.

You must enable third-party cookies to access PBX.

To allow third-party cookies:

  1. Open your browser Settings.
  2. Go to Privacy & security > Third-party cookies.
  3. Select Allow third-party cookies.
note

Browsers often block third-party cookies in Private or Incognito modes automatically.

Ensure your browser allows these specific cookies:

  • refresh-token
  • kameleoonSessionId
  • access-token
  • expiration-access

Why should I use Kameleoon PBX Ideate?​

Fewer than 20% of launched experiments yield positive results, often because teams test low-impact ideas or vague hypotheses. PBX Ideate provides prioritized, evidence-based hypotheses and converts them into precise prompts for PBX. For high-confidence recommendations, PBX Ideate achieves three times the accuracy of the industry benchmark, helping you prioritize your tests effectively.

How does PBX Ideate generate test ideas?​

PBX Ideate uses a taxonomy and semantic framework developed with Conversion by GAIN. It analyzes your website and uses data from tens of thousands of experiments to surface the ideas most likely to succeed.

How does PBX Ideate prioritize test ideas?​

PBX Ideate scores ideas based on potential impact using a model trained on over 20,000 experiments. This model provides 63% accuracy on high-confidence predictions, exceeding the industry average of about 20%.

Who can access PBX Ideate?​

Kameleoon prioritizes invited customers, but access is also open to invited partners. If you lack an invitation, you can join the waitlist. Kameleoon will add more spots gradually.

Do I need a Kameleoon account for PBX Ideate?​

Yes. If accepted, Kameleoon creates your account and emails registration instructions so you can begin testing.

Must I agree to specific terms for PBX Ideate?​

Participants must accept the beta program’s terms and conditions before receiving access. It requires no credit card.

How long does access to PBX Ideate last?​

Kameleoon grants 30-day access on a case-by-case basis. We will communicate updates regarding new features and general availability over time.

Must I install a script on my website to use PBX?​

You do not need to install a script to generate ideas or use PBX for front-end experiences. However, to run these experiments live on your site, you must install Kameleoon Web Experimentation.