USGS plugin not working

Start with Extract_USGS.py and the older get_usgs.py script named in the report, then trace how they call UsgsDataRetrieval.getData at the failing request. Check how the USGS data endpoint is accessed and compare it with the current API behavior. Done when retrieval succeeds for the reported station and the plugin can retrieve its station data.

It appears the USGS plugin that comes in RTS is using the old API, and I'm not able to get data anymore. We tried the old get_usgs.py script, and then tried the Extract_USGS.py that is in this repo. The logs showed 403 errors when trying to access the data via plugin, as shown below.

retrieve: thought recnum=3168 got=0
retrieve:urlStr=https://waterdata.usgs.gov/nwis/uv?format=rdb&period=33&site_no=06478513
IOException java.io.IOException: Server returned HTTP response code: 403 for URL: https://waterdata.usgs.gov/nwis/uv?format=rdb&period=33&site_no=06478513
java.io.IOException: Server returned HTTP response code: 403 for URL: https://waterdata.usgs.gov/nwis/uv?format=rdb&period=33&site_no=06478513
  at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1924)
  at java.base/sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1512)
  at java.base/sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1510)
  at java.base/java.security.AccessController.doPrivileged(Native Method)
  at java.base/java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:795)
  at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1509)
  at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:250)
  at hec.plugins.usgs.UsgsDataRetrieval.getData(UsgsDataRetrieval.java:362)
  at hec.plugins.usgs.UsgsDataRetrieval.retrieve(UsgsDataRetrieval.java:175)
  at hec.plugins.usgs.UsgsControlPanel.retrieveData(UsgsControlPanel.java:1066)
  at hec.plugins.usgs.UsgsControlFrame.retrieveData(UsgsControlFrame.java:761)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.base/java.lang.reflect.Method.invoke(Method.java:566)
  at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:190)
  at org.python.core.PyReflectedFunction.__call__(PyReflectedFunction.java:208)
  at org.python.core.PyObject.__call__(PyObject.java:461)
  at org.python.core.PyObject.__call__(PyObject.java:465)
  at org.python.core.PyMethod.__call__(PyMethod.java:126)
  at org.python.pycode._pyx14.f$0(<string>:35)
  at org.python.pycode._pyx14.call_function(<string>)
  at org.python.core.PyTableCode.call(PyTableCode.java:173)
  at org.python.core.PyCode.call(PyCode.java:18)
  at org.python.core.Py.runCode(Py.java:1703)
  at org.python.core.Py.exec(Py.java:1747)
  at org.python.util.PythonInterpreter.exec(PythonInterpreter.java:268)
  at hec.script.ScriptEngine.execute(ScriptEngine.java:698)
  at hec.script.ScriptEngine.execute(ScriptEngine.java:621)
  at hec2.rts.action.JythonScriptAction.runScript(JythonScriptAction.java:71)
  at hec2.rts.action.JythonScriptAction.actionPerformed(JythonScriptAction.java:55)
  at java.desktop/javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967)
  at java.desktop/javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308)
  at java.desktop/javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405)
  at java.desktop/javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262)
  at java.desktop/javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:279)
  at java.desktop/java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:297)
  at java.desktop/java.awt.Component.processMouseEvent(Component.java:6635)
  at java.desktop/javax.swing.JComponent.processMouseEvent(JComponent.java:3342)
  at java.desktop/java.awt.Component.processEvent(Component.java:6400)
  at java.desktop/java.awt.Container.processEvent(Container.java:2263)
  at java.desktop/java.awt.Component.dispatchEventImpl(Component.java:5011)
  at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2321)
  at java.desktop/java.awt.Component.dispatchEvent(Component.java:4843)
  at java.desktop/java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4918)
  at java.desktop/java.awt.LightweightDispatcher.processMouseEvent(Container.java:4547)
  at java.desktop/java.awt.LightweightDispatcher.dispatchEvent(Container.java:4488)
  at java.desktop/java.awt.Container.dispatchEventImpl(Container.java:2307)
  at java.desktop/java.awt.Window.dispatchEventImpl(Window.java:2772)
  at java.desktop/java.awt.Component.dispatchEvent(Component.java:4843)
  at java.desktop/java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772)
  at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:721)
  at java.desktop/java.awt.EventQueue$4.run(EventQueue.java:715)
  at java.base/java.security.AccessController.doPrivileged(Native Method)
  at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
  at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:95)
  at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:745)
  at java.desktop/java.awt.EventQueue$5.run(EventQueue.java:743)
  at java.base/java.security.AccessController.doPrivileged(Native Method)
  at java.base/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:85)
  at java.desktop/java.awt.EventQueue.dispatchEvent(EventQueue.java:742)
  at java.desktop/java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203)
  at java.desktop/java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124)
  at java.desktop/java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113)
  at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109)
  at java.desktop/java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
  at java.desktop/java.awt.EventDispatchThread.run(EventDispatchThread.java:90)
Error reading 06478513
retrieve: thought recnum=3168 got=0
0 out of 42 stations retrieved.