Why getting "TypeError: redeclaration of const e." while scraping with HtmlUnit?

huangapple
go评论76阅读模式
英文:

Why getting "TypeError: redeclaration of const e." while scraping with HtmlUnit?

问题

以下是您提供的代码的翻译部分：

private void htmlUnit() {

    String START_URL = "https://www.tradingview.com/symbols/BTCUSDT/?exchange=BINANCE";
    try {
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setCssEnabled(false);
        webClient.waitForBackgroundJavaScript(10000);
        HtmlPage page = webClient.getPage(START_URL);
        System.out.println();
    } catch (IOException ex) {
        ex.printStackTrace();
    }

}

以下是您提供的错误信息的翻译部分：

com.gargoylesoftware.htmlunit.ScriptException: TypeError: 重声明 const e。 (https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js#145)
    at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:883) [htmlunit-2.36.0.jar:2.36.0]
    ...
Caused by: net.sourceforge.htmlunit.corejs.javascript.EvaluatorException: TypeError: 重声明 const e。 (https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js#145)
    ...
英文:

I want to scrape live Bitcoin price using the HtmlUnit. I am running the following code to get the content of the website but getting an error.

private void htmlUnit() {

    String START_URL = "https://www.tradingview.com/symbols/BTCUSDT/?exchange=BINANCE";
    try {
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setCssEnabled(false);
        webClient.waitForBackgroundJavaScript(10000);
        HtmlPage page = webClient.getPage(START_URL);
        System.out.println();
    } catch (IOException ex) {
        ex.printStackTrace();
    }

}

Exception that I get:

      com.gargoylesoftware.htmlunit.ScriptException: TypeError: redeclaration of const e. (https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js#145)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:883) [htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:617) [htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:534) [htmlunit-core-js-2.36.0.jar:na]
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory.callSecured(HtmlUnitContextFactory.java:336) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.compile(JavaScriptEngine.java:714) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.compile(JavaScriptEngine.java:680) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.compile(JavaScriptEngine.java:103) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadJavaScriptFromUrl(HtmlPage.java:1102) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HtmlPage.loadExternalJavaScriptFile(HtmlPage.java:982) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HtmlScript.executeScriptIfNeeded(HtmlScript.java:362) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HtmlScript$2.execute(HtmlScript.java:236) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HtmlScript.onAllChildrenAddedToPage(HtmlScript.java:257) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:792) [htmlunit-2.36.0.jar:2.36.0]
at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) [xercesImpl-2.12.0.jar:na]
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.endElement(HTMLParser.java:748) [htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.callEndElement(HTMLTagBalancer.java:1175) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLTagBalancer.endElement(HTMLTagBalancer.java:1115) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.filters.DefaultFilter.endElement(DefaultFilter.java:219) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.filters.NamespaceBinder.endElement(NamespaceBinder.java:303) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scanEndElement(HTMLScanner.java:3146) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLScanner$ContentScanner.scan(HTMLScanner.java:2074) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLScanner.scanDocument(HTMLScanner.java:922) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:438) [neko-htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.cyberneko.HTMLConfiguration.parse(HTMLConfiguration.java:389) [neko-htmlunit-2.36.0.jar:2.36.0]
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) [xercesImpl-2.12.0.jar:na]
at com.gargoylesoftware.htmlunit.html.HTMLParser$HtmlUnitDOMBuilder.parse(HTMLParser.java:991) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HTMLParser.parse(HTMLParser.java:246) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.html.HTMLParser.parseHtml(HTMLParser.java:190) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createHtmlPage(DefaultPageCreator.java:269) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.DefaultPageCreator.createPage(DefaultPageCreator.java:160) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:534) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:400) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:317) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:469) [htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.WebClient.getPage(WebClient.java:450) [htmlunit-2.36.0.jar:2.36.0]
at simplespringbootapp.demo.DemoApplication.htmlUnit(DemoApplication.java:64) [classes/:na]
at simplespringbootapp.demo.DemoApplication.run(DemoApplication.java:47) [classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) [spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:779) [spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) [spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) [spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.3.4.RELEASE.jar:2.3.4.RELEASE]
at simplespringbootapp.demo.DemoApplication.main(DemoApplication.java:37) [classes/:na]
Caused by: net.sourceforge.htmlunit.corejs.javascript.EvaluatorException: TypeError: redeclaration of const e. (https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js#145)
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory$HtmlUnitErrorReporter.error(HtmlUnitContextFactory.java:420) ~[htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addError(Parser.java:259) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addError(Parser.java:235) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.defineSymbol(Parser.java:2273) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.variables(Parser.java:2172) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1225) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statements(Parser.java:1090) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.block(Parser.java:1994) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1249) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.getNextStatementAfterInlineComments(Parser.java:1496) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.forLoop(Parser.java:1607) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1198) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statements(Parser.java:1090) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.block(Parser.java:1994) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1249) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.getNextStatementAfterInlineComments(Parser.java:1496) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.ifStatement(Parser.java:1327) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1186) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.parseFunctionBody(Parser.java:772) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.function(Parser.java:942) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:3119) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2751) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2645) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2566) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2550) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2531) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2506) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2478) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2467) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2456) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2445) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2434) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2423) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2387) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2343) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2366) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.expr(Parser.java:2322) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.nameOrLabel(Parser.java:2071) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1271) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.parseFunctionBody(Parser.java:772) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.function(Parser.java:942) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:3119) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2751) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2645) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2566) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2550) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2531) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2506) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2478) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2467) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2456) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2445) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2434) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2423) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2387) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2343) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.plainProperty(Parser.java:3715) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.objectLiteral(Parser.java:3615) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:3127) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2751) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2645) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2566) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2550) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2531) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2506) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2478) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2467) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2456) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2445) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2434) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2423) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2387) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2343) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.arrayLiteral(Parser.java:3318) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.primaryExpr(Parser.java:3123) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2751) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2645) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2566) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2550) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2531) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2506) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2478) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2467) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2456) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2445) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2434) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2423) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2387) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2343) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.argumentList(Parser.java:2718) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExprTail(Parser.java:2868) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.memberExpr(Parser.java:2787) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.unaryExpr(Parser.java:2645) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.mulExpr(Parser.java:2566) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.addExpr(Parser.java:2550) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.shiftExpr(Parser.java:2531) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.relExpr(Parser.java:2506) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.eqExpr(Parser.java:2478) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitAndExpr(Parser.java:2467) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitXorExpr(Parser.java:2456) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.bitOrExpr(Parser.java:2445) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.andExpr(Parser.java:2434) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.orExpr(Parser.java:2423) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.condExpr(Parser.java:2387) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.assignExpr(Parser.java:2343) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.expr(Parser.java:2322) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statementHelper(Parser.java:1281) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.statement(Parser.java:1135) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.parse(Parser.java:672) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Parser.parse(Parser.java:593) ~[htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Context.parse(Context.java:2642) [htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Context.compileImpl(Context.java:2588) [htmlunit-core-js-2.36.0.jar:na]
at net.sourceforge.htmlunit.corejs.javascript.Context.compileString(Context.java:1582) [htmlunit-core-js-2.36.0.jar:na]
at com.gargoylesoftware.htmlunit.javascript.HtmlUnitContextFactory$TimeoutContext.compileString(HtmlUnitContextFactory.java:222) ~[htmlunit-2.36.0.jar:2.36.0]
at net.sourceforge.htmlunit.corejs.javascript.Context.compileString(Context.java:1571) [htmlunit-core-js-2.36.0.jar:na]
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$1.doRun(JavaScriptEngine.java:705) ~[htmlunit-2.36.0.jar:2.36.0]
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:868) [htmlunit-2.36.0.jar:2.36.0]
... 42 common frames omitted

答案1

得分: 2

你之所以遇到这个错误，主要是因为HtmlUnit使用的js引擎与当前浏览器的js不完全兼容。
虽然从版本到版本引擎得到了改进，但仍有很多工作要做。

更加技术性的解释是，请查看这个js文件：https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js。在这段代码中，某个地方可能是错误的原因……

你可以设置：

webClient.getOptions().setThrowExceptionOnScriptError(false);

以便让js引擎忽略错误并继续处理js代码。

最后一个要点是：

webClient.waitForBackgroundJavaScript(10000);

这不是一个选项。在页面获取后，你必须调用这个方法。

你的代码应该类似于这样：

String url = "https://www.tradingview.com/symbols/BTCUSDT/?exchange=BINANCE";
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
    webClient.getOptions().setThrowExceptionOnScriptError(false);

    HtmlPage page = webClient.getPage(url);
    webClient.waitForBackgroundJavaScript(20_000);

    System.out.println(" ---- ");
    System.out.println(page.asText());
    System.out.println(" ---- ");
}

希望这对你有帮助。请记住，至少图表是基于画布渲染的，这意味着你可以使用asText()方法获取它。

英文:

You got this error more or less because the js engine used by HtmlUnit is not 100% compatible with the js of current browsers.
The engine gets improved from version to version but there is still a lot to do.

Or more technical - have a look at this js file https://www.tradingview.com/static/bundles/category.1bca33621fc200171fb7.js. Somewhere in this code is the reason for the error....

You can set

webClient.getOptions().setThrowExceptionOnScriptError(false);

to let the js engine ignore the error and process the js code further.

And one last point

webClient.waitForBackgroundJavaScript(10000);

is not an option. You have to call this after the page retrieval.

Your code should look like this

String url = "https://www.tradingview.com/symbols/BTCUSDT/?exchange=BINANCE";
try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX)) {
webClient.getOptions().setThrowExceptionOnScriptError(false);
HtmlPage page = webClient.getPage(url);
webClient.waitForBackgroundJavaScript(20_000);
System.out.println(" ---- ");
System.out.println(page.asText());
System.out.println(" ---- ");
}

Hope this helps. And please keep in mind at least the chart is rendered based on a canvas - means you can get this with asText().

答案2

得分: 1

我编写用于HTML抓取的软件。我的工具用于生成外国新闻服务的翻译。这是为了解释此答案中的背景信息——这样您就可以理解**HTML抓取**在何时最有用，以及何时最好避免使用它（并且可能效率低下且“有点”不必要）。

在提供REST API的情况下，几乎总是更“明智”使用**REST API。这篇维基百科关于REST API的文章包含下面复制（剪切和粘贴）的描述。我看过或了解过的几乎所有REST都会以web服务器查询的形式返回JSON**作为响应。

> 遵循REST架构约束的Web服务API被称为RESTful API。基于HTTP的RESTful API具有以下方面：
> - 基本URI，例如http://api.example.com/collection/；标准HTTP方法（例如GET、POST、PUT、PATCH和DELETE）；
> - 定义状态转换数据元素的媒体类型（例如Atom、microformats、application/vnd.collection+json等）。
> - 当前表示告诉客户端如何组合请求以转换到所有下一个可用的应用程序状态。这可能是一个简单的URI，也可能是一个复杂的Java小程序。

通常了解如何使用JSON API需要使用Java拥有的两种不同的JSON解析器之一。其中之一被埋在**javax.json包层次结构中，另一个位于android.JSON**开发平台包中。无论哪种都能帮助您解析REST API的响应，而这些几乎总是比尝试在充满JavaScript的网站上运行JavaScript要好得多。

这里是JSON解析的文档站点 - 再次强调Java有两个常用的解析器：

我提到我使用的HTML解析器，恰好是因为我试图解释，在唯一获取所需数据的方法是通过**HTML解析**（甚至是执行Java脚本）的情况下，但是当实际的API提供给“广大互联网”时，最好使用这些API。

通过Google快速搜索可以找到数十个用于比特币交易者的**REST API** - 使用我提到的任一JSON库之一来解析这些响应将更明智。这些链接直接从在**Google搜索栏**中输入“BitCount REST API”而来。

> 您正在尝试使用**Selenium抓取的页面是我见过的较复杂的Java脚本/ AJAX加载页面之一。对Selenium来说，执行该脚本可能会很困难。在我的生活中，我犯过很多错误，但我想建议您使用不同的Web URL**来获取比特币价格。

英文:

I write software for HTML Scraping. My tools are for generating translations of foreign news-services. This is being mentioned to explain the background information in this answer - so you may understand where HTML Scrape is most useful, and when it is better avoided (and likely inefficient and 'kind of' unnecessary).

In cases where a REST API is provided, it almost always "wiser" to use the REST API. This Wikipedia Article about REST API's contains this below copied (cut and paste) description of what they are. Almost all of the REST's that I have seen or read about provide JSON as a response to a web-server query.

> Web service APIs that adhere to the REST architectural constraints are
> called RESTful APIs.[14] HTTP-based RESTful APIs are defined with the
> following aspects:[15]
>
> - a base URI, such as http://api.example.com/collection/; standard HTTP methods (e.g., GET, POST, PUT, PATCH and DELETE);
> - a media type that defines state transition data elements (e.g., Atom, microformats, application/vnd.collection+json,[15]:91–99 etc.).
> - The current representation tells the client how to compose requests for transitions to all the next available application states. This
> could be as simple as a URI or as complex as a Java applet.[16]

Generally understanding how to use a JSON API requires using one of two different JSON Parser's that Java has at its disposal. One of them is buried in the javax.json package hierarchy-tree, and the other is in the android.JSON development platform package. Either one is will help you parse the responses to REST API's, and those are nearly always better than trying to run Java-Script on web-sites that are filled with Java-Script.

Here are the documentation sites for JSON Parsing - again Java has two that are popularly used:

I mention the HTML Parser I use precisely because I try to explain that there are circumstances where the only way to obtain the data one needs is by HTML Parsing (and even Java-Script execution), however, when an actual API is provided to the "Internet at Large" it is much better to use those instead.

A quick search on Google reveals dozens of REST API's for BitCoin Traders - and experimenting with parsing those responses using either of the JSON libraries I have mentioned would be smarter. These links were copied directly from entering "BitCount REST API" into the Google Search Bar

> The page you are trying to scrape using Selenium is one of the
> more complicated Java-Script / AJAX laden pages I have seen. It might
> be difficult for Selenium to execute that script easily. I have
> been wrong many times in my life, but I thought I might suggest using
> a different Web URL for BitCoin Prices

huangapple
  • 本文由 发表于 2020年9月26日 20:39:30
  • 转载请务必保留本文链接：https://go.coder-hub.com/64077779.html
  • htmlunit
  • java
  • web-scraping
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定