How to add args to goog:chromeOptions when creating a chromeDriver?

[dunand]

When creating a webdriver. I need the possibility to add arguments. Like the two proxy args below:

case "googlechromeheadless":
   desiredCapabilities = new ChromeOptions();
   logging.debug("Parsing chrome options: "+browserOptions);
   parseBrowserOptionsChrome(browserOptions, desiredCapabilities);
   ((ChromeOptions)desiredCapabilities).setHeadless(true);
   ((ChromeOptions)desiredCapabilities).addArguments("--proxy-server='direct://'", "--proxy-bypass-list=*"); 
   break;

How can I add the goog:chromeOptions args without modifying the source code?

When I look at the chromeDriver log, I need to see something like this:

"desiredCapabilities": {
         "acceptInsecureCerts": true,
         "browserName": "chrome",
         "goog:chromeOptions": {
            "args": [ "--proxy-server=direct://", "--proxy-bypass-list=*" ],
            "extensions": [  ]
         }
      }

[dunand]

I copied the modified version of BrowserManagement.java that include modifications to support addition of goog:chromeOptions args :

 @RobotKeyword("Opens a new browser instance to the given ``url``.\r\n" + 
            "\r\n" + 
            "The ``browser`` argument specifies which browser to use, and the supported browser are listed in the table below. The browser names are case-insensitive and some browsers have multiple supported names.\r\n" + 
            "|    = Browser =    | = Name(s) =       |\r\n" + 
            "| Firefox   | firefox, ff      |\r\n" + 
            "| Firefox (headless)   | firefoxheadless, ffheadless      |\r\n" + 
            "| Google Chrome     | googlechrome, chrome, gc |\r\n" +
            "| Google Chrome (headless)    | googlechromeheadless, chromeheadless, gcheadless |\r\n" + 
            "| Internet Explorer | internetexplorer, ie     |\r\n" + 
            "| Edge      | edge      |\r\n" + 
            "| Safari    | safari    |\r\n" + 
            "| Opera     | opera     |\r\n" + 
            "| Android   | android   |\r\n" + 
            "| Iphone    | iphone    |\r\n" + 
            "| PhantomJS | phantomjs |\r\n" + 
            "| HTMLUnit  | htmlunit  |\r\n" + 
            "| HTMLUnit with Javascript | htmlunitwithjs    |\r\n" + 
            "| JBrowser  | jbrowser  |\r\n" + 
            "\r\n" + 
            "To be able to actually use one of these browsers, you need to have a matching Selenium browser driver available. See the [https://github.com/Hi-Fi/robotframework-seleniumlibrary-java#browser-drivers|project documentation] for more details.\r\n" + 
            "\r\n" + 
            "Optional ``alias`` is an alias given for this browser instance and it can be used for switching between browsers. An alternative approach for switching is using an index returned by this keyword. These indices start from 1, are incremented when new browsers are opened, and reset back to 1 when `Close All Browsers` is called. See `Switch Browser` for more information and examples.\r\n" + 
            "\r\n" + 
            "Optional ``remote_url`` is the URL for a remote Selenium server. If you specify a value for a remote, you can also specify ``desired_capabilities`` to configure, for example, a proxy server for Internet Explorer or a browser and operating system when using [http://saucelabs.com|Sauce Labs]. Desired capabilities can be given as a dictionary. [https://github.com/SeleniumHQ/selenium/wiki/Capabilities| Selenium documentation] lists possible capabilities that can be enabled.\r\n" + 
            "\r\n" + 
            "Optional ``ff_profile_dir`` is the path to the Firefox profile directory if you wish to overwrite the default profile Selenium uses. Notice that prior to SeleniumLibrary 3.0, the library contained its own profile that was used by default.\r\n" + 
            "\r\n" + 
            "Examples:\r\n" + 
            "| `Open Browser` | http://example.com | Chrome  |\r\n" + 
            "| `Open Browser` | http://example.com | Firefox | alias=Firefox |\r\n" + 
            "| `Open Browser` | http://example.com | Edge    | remote_url=http://127.0.0.1:4444/wd/hub |\r\n" + 
            "\r\n" + 
            "If the provided configuration options are not enough, it is possible to use `Create Webdriver` to customize browser initialization even more.")
    @ArgumentNames({ "url", "browserName=firefox", "alias=None", "remoteUrl=None", "desiredCapabilities=None",
            "browserOptions=None", "args=None" })
    public String openBrowser(String url, String... args) throws Throwable {
        String browserName = robot.getParamsValue(args, 0, "firefox");
        String alias = robot.getParamsValue(args, 1, "None");
        String remoteUrl = robot.getParamsValue(args, 2, "None");
        String desiredCapabilities = robot.getParamsValue(args, 3, "None");
        String browserOptions = robot.getParamsValue(args, 4, "None");
        String arguments = robot.getParamsValue(args, 5, "");

        try {
            logging.info("browserName: " + browserName);

            WebDriver webDriver = createWebDriver(browserName, desiredCapabilities, remoteUrl, browserOptions, arguments);
            webDriver.get(url);
            String sessionId = webDriverCache.register(webDriver, alias);
            logging.debug(String.format("Opened browser with session id %s", sessionId));
            return sessionId;
        } catch (Throwable t) {
            if (remoteUrl != null && !remoteUrl.equalsIgnoreCase("FALSE") && !remoteUrl.equalsIgnoreCase("NONE")) {
                logging.warn(String.format("Opening browser '%s' to base url '%s' through remote server at '%s' failed",
                        browserName, url, remoteUrl));
            } else {
                logging.warn(String.format("Opening browser '%s' to base url '%s' failed", browserName, url));
            }
            throw new SeleniumLibraryFatalException(t);
        }
    }

...

    protected WebDriver createWebDriver(String browserName, String desiredCapabilitiesString, String remoteUrlString,
            String browserOptions, String args) throws MalformedURLException {
        browserName = browserName.toLowerCase().replace(" ", "");
        Capabilities desiredCapabilities = createCapabilities(browserName, desiredCapabilitiesString,
                browserOptions, args);

        WebDriver webDriver;
        if (remoteUrlString != null && !remoteUrlString.equalsIgnoreCase("FALSE") && !remoteUrlString.equalsIgnoreCase("NONE")) {
            logging.info(String.format("Opening browser '%s' through remote server at '%s'",
                    browserName, remoteUrlString));
            webDriver = createRemoteWebDriver(desiredCapabilities, new URL(remoteUrlString));
        } else {
            logging.info(String.format("Opening browser '%s'", browserName));
            webDriver = createLocalWebDriver(browserName, desiredCapabilities);
        }

        webDriver.manage().timeouts().setScriptTimeout((int) (timeout * 1000.0), TimeUnit.MILLISECONDS);
        webDriver.manage().timeouts().implicitlyWait((int) (implicitWait * 1000.0), TimeUnit.MILLISECONDS);

        return webDriver;
    }

...

    protected Capabilities createCapabilities(String browserName, String desiredCapabilitiesString,
            String browserOptions, String args) {
        Capabilities desiredCapabilities;
        switch (browserName.toLowerCase()) {
        case "ff":
        case "firefox":
            desiredCapabilities = new FirefoxOptions();
            parseBrowserOptionsFirefox(browserOptions, desiredCapabilities);
            break;
        case "ffheadless":
        case "firefoxheadless":
            desiredCapabilities = new FirefoxOptions();
            parseBrowserOptionsFirefox(browserOptions, desiredCapabilities);
            ((FirefoxOptions)desiredCapabilities).setHeadless(true);
            break;
        case "ie":
        case "internetexplorer":
            desiredCapabilities = new InternetExplorerOptions();
            break;
        case "edge":
            desiredCapabilities = new EdgeOptions();
            break;
        case "gc":
        case "chrome":
        case "googlechrome":
            desiredCapabilities = new ChromeOptions();
            logging.debug("Parsing chrome options: "+browserOptions);
            parseBrowserOptionsChrome(browserOptions, desiredCapabilities);
            parseChromeArgs((ChromeOptions)desiredCapabilities, args);
            break;
        case "gcheadless":
        case "chromeheadless":
        case "googlechromeheadless":
            desiredCapabilities = new ChromeOptions();
            logging.debug("Parsing chrome options: "+browserOptions);
            parseBrowserOptionsChrome(browserOptions, desiredCapabilities);
            ((ChromeOptions)desiredCapabilities).setHeadless(true);
            parseChromeArgs((ChromeOptions)desiredCapabilities, args);
            //((ChromeOptions)desiredCapabilities).addArguments(args);  // "--proxy-server='direct://'", "--proxy-bypass-list=*"
            break;
        case "opera":
            desiredCapabilities = new OperaOptions();
            break;
        case "phantomjs":
            desiredCapabilities = DesiredCapabilities.phantomjs();
            break;
        case "safari":
            desiredCapabilities = new SafariOptions();
            break;
        case "htmlunit":
        case "htmlunitwithjs":
            desiredCapabilities = DesiredCapabilities.htmlUnit();
            ((DesiredCapabilities) desiredCapabilities).setBrowserName("htmlunit");
            break;
        case "jbrowser":
            desiredCapabilities = new DesiredCapabilities("jbrowser", "1", Platform.ANY);
            break;
        default:
            throw new SeleniumLibraryFatalException(browserName + " is not a supported browser.");
        }

        if (desiredCapabilitiesString != null && !"None".equals(desiredCapabilitiesString)) {
            JSONObject jsonObject = (JSONObject) JSONValue.parse(desiredCapabilitiesString);
            if (jsonObject != null) {
                // Valid JSON
                Iterator<?> iterator = jsonObject.entrySet().iterator();
                while (iterator.hasNext()) {
                    Entry<?, ?> entry = (Entry<?, ?>) iterator.next();
                    ((MutableCapabilities) desiredCapabilities).setCapability(entry.getKey().toString(), entry.getValue());
                }
            } else {
                // Invalid JSON. Old style key-value pairs
                for (String capability : desiredCapabilitiesString.split(",")) {
                    String[] keyValue = capability.split(":");
                    if (keyValue.length == 2) {
                        ((MutableCapabilities) desiredCapabilities).setCapability(keyValue[0], keyValue[1]);
                    } else {
                        logging.warn("Invalid desiredCapabilities: " + desiredCapabilitiesString);
                    }
                }
            }
        }
        return desiredCapabilities;
    }

...

    protected void parseChromeArgs(ChromeOptions chromeOptions, String args) {
        if (args != null && !"NONE".equalsIgnoreCase(args)) {
            JSONObject jsonObject = (JSONObject) JSONValue.parse(args);
            if (jsonObject != null) {
                Iterator<?> iterator = jsonObject.entrySet().iterator();
                while (iterator.hasNext()) {
                    Entry<?, ?> entry = (Entry<?, ?>) iterator.next();
                    String key = entry.getKey().toString();
                    String value = entry.getValue().toString();
                    logging.debug(String.format("Adding args: %s with value: %s", key, value));
                    chromeOptions.addArguments("--"+key+"="+value);
                }
            } else {
                logging.warn("Invalid args: " + args);
            }
        }
    }

[Hi-Fi]

At least in example those arguments you mentioned are also allowed to be put in command line (https://peter.sh/experiments/chromium-command-line-switches/), and there's already an way to put those in as arguments list within browserOptions. E.g.

${browserOptions}    Set Variable    {"args":[ "--proxy-server=direct://", "--proxy-bypass-list=*" ],"extensions":[],"prefs":{}}
SeleniumLibrary.Open Browser    ${targetURL}    ${browser}    browserOptions=${browserOptions}

In Python version there's also more low level way to create a webdriver, which would be the way to go within this also if functionality is really needed. But for example I think the browserOptions is the way to go.

[dunand]

The ${browserOptions} does not work because the args parameter is overwrite in the asMap method of the ChromeOptions class:

  public Map<String, Object> asMap() {
      ...
      options.put("args", ImmutableList.copyOf(args));  
      ...
      // before this line toReturn contain: 
      // {browserName=chrome, goog:chromeOptions={args=["--proxy-server=direct:\/\/","--proxy-bypass-list=*"], extensions=[], prefs={}}}

      toReturn.put(CAPABILITY, options);

      // after this line toReturn contain:
      // {browserName=chrome, goog:chromeOptions={args=[], extensions=[]}}

[Hi-Fi]

How you're calling that? At least there's a test that checks that those should be in place before passing to browser in here.

At the Robot script side there should be no mention of goog:chromeOptions, as that's value is at the "browserOptions" string.

[Hi-Fi]

Also when adding to the test values from your example:

    @Test
    public void parseChromeBrowserOptions() {
        ChromeOptions chromeOptions = new ChromeOptions();
        String browserOptions = " {\n" +
                "            \"args\": [ \"--proxy-server=direct://\", \"--proxy-bypass-list=*\" ],\n" +
                "            \"extensions\": [  ]\n" +
                "         }";
        bm.parseBrowserOptionsChrome(browserOptions, chromeOptions);
        System.out.println(chromeOptions);
    }

Print out is:
Capabilities {browserName: chrome, goog:chromeOptions: {args: [--proxy-server=direct://, --proxy-bypass-list=*], extensions: []}}

[dunand]

I'm calling it like that:

	${browserOptions}    Set Variable    {"args":[ "--proxy-server=direct://", "--proxy-bypass-list=*" ],"extensions":[],"prefs":{}}
	Open Browser  ${test.url}/lel/fr/miser/lottoMax  chromeheadless  None  None  ${test.browserCapabilities}  ${browserOptions} 

I looked at your test and this is working fine for me too. The parsing of the args is working. It's after that in the ChromeOptions.asMap method that the args parameter is overwrite.

See by yourself if you activate logging for the chrome driver:

-Dwebdriver.chrome.logfile=C:/selenium/chromedriver.log -Dwebdriver.chrome.verboseLogging=true

You will look at the log and the args parameter will only contains the headless parameters like that:

"desiredCapabilities": {
         "acceptInsecureCerts": true,
         "browserName": "chrome",
         "goog:chromeOptions": {
            "args": [ "--headless", "--disable-gpu" ],
            "extensions": [  ]
         }
      }

Thanks for your time

[Hi-Fi]

Thank you, i'll check that.

For the original issue I think there's not going to be change, but this bug losing the arguments need to be fixed

[dunand]

If you fix the losing the arguments that will solve my problem.

Thank you

[Hi-Fi]

Seems that this was working fine with Selenium 3.8.1, but didn't work with 3.9.1 anymore. 3.9.1.0 was first version to use those browser specific Options instead of plain defiredCapabilities, so it might be that I'm just using those somehow wrong way.

[Hi-Fi]

Found bug at least at my end. The capabilities parsing to ChromeOptions doesn't move e.g. args to args at the Chromeoptions, which is why those never end up in started browser.

[Hi-Fi]

Deploy to Maven central made, might take a while to be visible. Thank you for pointing this issue out.