Merged PRs

dolt

• 8969: [statspro] stats ignore non prolly indexes
  Stats collection skips non-prolly indexes. A more complete/safe refactoring of index use is in the new stats PR.
  fixes: #8964
• 8963: store/nbs: s3_object_reader: Fix a regression which prevented loading large table files with the AWS NomsBlockStore backend.
  Affects 1.50.0, 1.50.1 and 1.50.2.
• 8962: Backup restore procedure can load aws config
• 8950: go: cmd/dolt: sqlserver: Create a Config struct to encapsulate arguments that control sqlserver.ConfigureServices behavior.
  Allow configuring the ProtocolListenerFactory through ConfigureServices.
• 8948: go: cmd/dolt: sqlserver: When stopping the svcs.Controller for the sql-server, drain client connections so that the handler is guaranteed to no longer be handling any connections when the server is stopped.
• 8944: Added documentation for auto_gc_behavior
• 8941: chore: added log_format for auto_generated yaml
  Added log_format in sqlserver.go to automate addition of log_format in config
• 8934: /integration-tests/MySQLDockerfile: run the rustup install shit
• 8931: fix use info schema panic
  The active database being information_schema causes some issues with branch state/working sets.
  gms side: dolthub/go-mysql-server#2873
  fixes: #8930 (comment)
• 8920: Sql debug cmd
  use:

  > dolt debug --output debug --time 10 -q "select * from table"
  starting cpu profile...
  starting mem profile...
  starting trace profile...
  debug results in: /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989
  zipped results in: debug.tar.gz
  > ls /var/folders/f7/0hsvdb9j0q71y3742k64jfn80000gn/T/dolt-debug-1522503989
  analysis.txt        exec.txt       mem.pprof               trace.out
  cpu.pprof           input.sql     plan.txt
  

  • cpu.pprof = CPU profile, run for at least time seconds (query loops if necessary)
  • mem.pprof = memory profile, run for at least time seconds (query loops if necessary)
  • trace.out = trace profile, run for at least time seconds (query loops if necessary)
  • input.sql = input query (select * from table)
  • analysis.txt = debug analysis output, includes memo and index cost estimations
  • plan.txt = simple and verbose plan output
  • exec.txt = accumulated stdout/stderr for all queries/background work
    note:
  • fails if server mode is running, having this be a stored procedure would be quite a bit more work
  • lightly tested, queries with errors, larger scripts with DML/DDL probably do not profile smoothly
• 8902: go/store/nbs: Move to golang maps instead of dolthub/swiss.
• 8890: feat:(logging) Added Log Format provision
  Added log_format feature for logger.

go-mysql-server

• 2885: [planbuilder] sort expression aliases always referenced
  This fixes a bug where a sort expression alias computed in the lower projection fails to index the nested expression.
  Below, the first plan's sort searches for c5:6 in the child, but only finds a:7. The second plan fixes the correctness issue. Obviously there are more desirable projection organizations that version two, but this is small enough of an edge case that I think rewriting projection management with proper expression interning would be overkill right now. The rest of the plan tests look OK/improvements.

  select distinct abs(c5) as a from one_pk where c2 in (1,11,31) order by a
  before:
  Sort(abs(one_pk.c5:6)->a:7 ASC nullsFirst)
  └─ Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:1)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:0
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [c2 c5]
  after:
  Distinct
  └─ Project
  ├─ columns: [abs(one_pk.c5:5)->a:0]
  └─ Sort(a:6 ASC nullsFirst)
  └─ Project
  ├─ columns: [one_pk.pk:0!null, one_pk.c1:1, one_pk.c2:2, one_pk.c3:3, one_pk.c4:4, one_pk.c5:5, abs(one_pk.c5:5)->a:0]
  └─ Filter
  ├─ HashIn
  │   ├─ one_pk.c2:2
  │   └─ TUPLE(1 (tinyint), 11 (tinyint), 31 (tinyint))
  └─ ProcessTable
  └─ Table
  ├─ name: one_pk
  └─ columns: [pk c1 c2 c3 c4 c5]

• 2883: allow before insert trigger to specify missing column, but better
  It is possible that a BEFORE INSERT TRIGGER fills in the values for a column when it is otherwise missing. This should not trigger a missing default value error.
  Fixes: #8926
  This PR is an improvement on: dolthub/go-mysql-server#2876
• 2881: server/handler.go: Improve some edge cases and error handling in resultForDefaultIter.
• 2879: server: Get rid of globals for setting a protocol listener factory. Get rid of unused, global-ridden and complicated Interceptor and Option functionality.
• 2878: server/context.go: Add a method on SessionManager which allows an integrator to wait for all client connections to drain.
• 2870: fix stored procedures in triggers
  When prepending nodes for trigger execution, we don't prepend to the *plan.Procedure member in the *plan.Call nodes.
• 2868: Don't use non-unique indexes as functional dependencies in AND expressions.
  This fixes a customer issue that was causing Error 1105 (HY000): result max1Row iterator returned more than one row
  We can probably do better than this: we could tweak it so that adding a FD key on a non-unique index instead adds an implication that the non-unique index key determines the primary key.
  But this is a simpler fix in the meantime that makes us more correct.

vitess

• 401: Revert moving call to recycleReadPacket()
  Dolt binlog tests started failing, due to the change in location of calling recycleReadPacket(). This change moves those two calls back to their original location.
• 400: Cleanup: fixing buffer reuse issues and log statements
• 399: go/mysql: conn.go: Fix read-after-recycle bug of the packet byte buffer in COM_{STMT_SEND_LONG_DATA,REGISTER_REPLICA,BINLOG_DUMP_GTID}.
  Back ports vitessio/vitess@24820d8

Closed Issues

• 8964: Recurrent Panic with querying geometry types after some queries
• 8926: BUG: BEFORE UPDATE Trigger Not Working in Dolt
• 8952: Update Roadmap
• 8930: Panic when browsing database tables after connecting to INFORMATION_SCHEMA via SQLTools extension
• 8888: Feature Request: Json Logging
• 8906: Feature Request: Add support for REGEXP_SUBSTR() and REGEXP_INSTR() functions