How to connect to sql server

Open
#76 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
docker, ruby, sql
Domain
database, devops

Research direction

Start by reading the provided Dockerfile and fluent.conf, then reproduce the adapter-loading errors in the stated Fluentd container setup. Trace how the fluent-plugin-sql configuration loads tiny_tds or activerecord-sqlserver-adapter; done means the configured adapter loads and the SQL match can connect to SQL Server.

Written by the indexing model from the issue text.

Description

enhancement

I have tried installing tiny_tds, also tried activerecord-sqlserver-adapter but both give me the following error:

2019-10-14 07:21:34 +0000 [error]: #0 fluent/log.rb:362:error: Could not load the 'tiny_tds' Active Record adapter. Ensure that the adapter is spelled correctly in config/database.yml and that you've added the necessary adapter gem to your Gemfile. path=nil error_class=LoadError error="Could not load the 'tiny_tds' Active Record adapter. Ensure that the adapter is spelled correctly in config/database.yml and that you've added the necessary adapter gem to your Gemfile."
2019-10-14 07:21:34 +0000 [error]: #0 fluent/log.rb:362:error: Could not load the 'activerecord-sqlserver-adapter' Active Record adapter. Ensure that the adapter is spelled correctly in config/database.yml and that you've added the necessary adapter gem to your Gemfile. path=nil error_class=LoadError error="Could not load the 'activerecord-sqlserver-adapter' Active Record adapter. Ensure that the adapter is spelled correctly in config/database.yml and that you've added the necessary adapter gem to your Gemfile."

Dockerfile:

# AUTOMATICALLY GENERATED
# DO NOT EDIT THIS FILE DIRECTLY, USE /Dockerfile.template.erb

FROM alpine:3.9
LABEL maintainer "Fluentd developers <fluentd@googlegroups.com>"
LABEL Description="Fluentd docker image" Vendor="Fluent Organization" Version="1.7.3"

# Do not split this into multiple RUN!
# Docker creates a layer for every RUN-Statement
# therefore an 'apk delete' has no effect
RUN apk update \
 && apk add --no-cache \
        ca-certificates linux-headers \
        ruby ruby-irb ruby-etc ruby-webrick \
        tini \
 && apk add --no-cache --virtual .build-deps \
        build-base \
        ruby-dev gnupg \
 && apk add freetds freetds-dev \
 && echo 'gem: --no-document' >> /etc/gemrc \
 && gem install tiny_tds -v 2.1.0 \
 && gem install fluent-plugin-sql -v 1.1.1 \
 && gem install activerecord-sqlserver-adapter -v 5.2.0 \
 && gem install oj -v 3.3.10 \
 && gem install json -v 2.2.0 \
 && gem install async-http -v 0.46.3 \
 && gem install fluentd -v 1.6.3 \
 && gem install bigdecimal -v 1.3.5 \
 && apk del .build-deps \
 && rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

RUN addgroup -S fluent && adduser -S -g fluent fluent \
    # for log storage (maybe shared with host)
    && mkdir -p /fluentd/log \
    # configuration/plugins path (default: copied from .)
    && mkdir -p /fluentd/etc /fluentd/plugins \
    && chown -R fluent /fluentd && chgrp -R fluent /fluentd


COPY fluent.conf /fluentd/etc/
COPY entrypoint.sh /bin/


ENV FLUENTD_CONF="fluent.conf"

ENV LD_PRELOAD=""
EXPOSE 24224 5140

USER fluent
ENTRYPOINT ["tini",  "--", "/bin/entrypoint.sh"]
CMD ["fluentd"]

conf

<source>
  @type forward
  port 24224
  bind 0.0.0.0
</source>

<match **>
  @type stdout
</match>
<match *>
  @type sql
  host ...
  port 1433
  database fluentd
  adapter tiny_tds / activerecord-sqlserver-adapter
  username ...
  password ...
  
  <table>
    table dbo.logs
    column_mapping 'timestamp:created_at,fluentdata1:dbcol1,fluentdata2:dbcol2,fluentdata3:dbcol3'
    # This is the default table because it has no "pattern" argument in <table>
    # The logic is such that if all non-default <table> blocks
    # do not match, the default one is chosen.
    # The default table is required.
  </table>

</match>
Dominant language
Ruby
Stars
112
Forks
58
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fluent/fluent-plugin-sql

All issues in fluent/fluent-plugin-sql

Similar issues

More Ruby issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.