Experiencing Exception on Hive/SASL/auth-conf

Start in pyhive/hive.py at sasl_factory and the GSSAPI setup shown in the traceback. Check how auth-conf frames are fetched, then make the maxbufsize behavior handle the reported 300-row query without sasl_decode failing. Verify the same fetch path against a Kerberos Hive connection using auth-conf.

When accessive Hive through SQLAlchemy or the Jupyter Hive Kernel, using Kerberos and the auth-conf setting, we are getting the following exception.

It occurs on a simple select statement listing 300 rows:

Traceback (most recent call last):
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1211, in fetchall
    l = self.process_rows(self._fetchall_impl())
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1161, in _fetchall_impl
    return self.cursor.fetchall()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 136, in fetchall
    return list(iter(self.fetchone, None))
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 105, in fetchone
    self._fetch_while(lambda: not self._data and self._state != self._STATE_FINISHED)
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 45, in _fetch_while
    self._fetch_more()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/hive.py", line 386, in _fetch_more
    response = self._connection.client.FetchResults(req)
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 714, in FetchResults
    return self.recv_FetchResults()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 726, in recv_FetchResults
    (fname, mtype, rseqid) = iprot.readMessageBegin()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 134, in readMessageBegin
    sz = self.readI32()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 217, in readI32
    buff = self.trans.readAll(4)
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
    chunk = self.read(sz - have)
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 173, in read
    self._read_frame()
  File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 187, in _read_frame
    message=self.sasl.getError())
thrift.transport.TTransport.TTransportException: b'Error in sasl_decode (-1) SASL(-1): generic failure: Unable to find a callback: 32775'

Turns out this is generated by the Cyrus-SASL native library.
I am able to fix the issue by specifying a larger value for 'maxbufsize' in the sasl_factory in hive.py. The default is 65535 bytes or so.

def sasl_factory():
                    sasl_client = sasl.Client()
                    sasl_client.setAttr('host', host)
                    if sasl_auth == 'GSSAPI':
                        sasl_client.setAttr('service', kerberos_service_name)
                        #+B. Gagnon 2021-07-12: patch to support auth-conf
                        sasl_client.setAttr('maxbufsize', 2*1024*1024)
                        #-B. Gagnon 2021-07-12: patch to support auth-conf
                    elif sasl_auth == 'PLAIN':
                        sasl_client.setAttr('username', username)
                        sasl_client.setAttr('password', password)
                    else:
                        raise AssertionError
                    sasl_client.init()
                    return sasl_client
                self._transport = thrift_sasl.TSaslClientTransport(sasl_factory, sasl_auth, socket)