cannot restart Percona server

Viewed 41
2022-09-09T17:07:48.919934Z 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 20162)
2022-09-09T17:07:48.920103Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 10000)
2022-09-09T17:07:49.064332Z 0 [Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
2022-09-09T17:07:49.064350Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
2022-09-09T17:07:49.065052Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.35-38) starting as process 19848 ...
2022-09-09T17:07:49.068463Z 0 [Note] InnoDB: PUNCH HOLE support available
2022-09-09T17:07:49.068488Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2022-09-09T17:07:49.068493Z 0 [Note] InnoDB: Uses event mutexes
2022-09-09T17:07:49.068497Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2022-09-09T17:07:49.068499Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2022-09-09T17:07:49.068502Z 0 [Note] InnoDB: Using Linux native AIO
2022-09-09T17:07:49.068676Z 0 [Note] InnoDB: Number of pools: 1
2022-09-09T17:07:49.068743Z 0 [Note] InnoDB: Using CPU crc32 instructions
2022-09-09T17:07:49.070138Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2022-09-09T17:07:49.104238Z 0 [Note] InnoDB: Completed initialization of buffer pool
2022-09-09T17:07:49.116836Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2022-09-09T17:07:49.129383Z 0 [Note] InnoDB: Recovering partial pages from the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite
2022-09-09T17:07:49.141427Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2022-09-09T17:07:49.151423Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 410343119828
2022-09-09T17:07:49.174173Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 410344579779
2022-09-09T17:07:49.174486Z 0 [Note] InnoDB: Database was not shutdown normally!
2022-09-09T17:07:49.174492Z 0 [Note] InnoDB: Starting crash recovery.
2022-09-09T17:07:49.230926Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite, size 31457280 bytes
2022-09-09T17:07:49.303505Z 0 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 17:07:49 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
Please help us make Percona Server better by reporting any
bugs at https://bugs.percona.com/

key_buffer_size=314572800
read_buffer_size=131072
max_used_connections=0
max_threads=152
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 637938 K  bytes of memory
Hope thats ok; if not, decrease some variables in the equation.


Build ID: 33c8eb1b0ac2d6905fbf2e1e6c6c6a31a2976eb3
Server Version: 5.7.35-38 Percona Server (GPL), Release '38', Revision '3692a61'

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
87 /usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x55e2c4f4280c]
88 89 /usr/sbin/mysqld(handle_fatal_signal+0x3f6)[0x55e2c4d727f6]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0)[0x7fe0842fe0e0]
90 91 /usr/sbin/mysqld(_Z23page_cur_insert_rec_lowPhP12dict_index_tPKhPmP5mtr_t+0x360)[0x55e2c5082c80]
92 /usr/sbin/mysqld(_Z25page_cur_parse_insert_recmPKhS0_P11buf_block_tP12dict_index_tP5mtr_t+0x660)[0x55e2c50848b0]
93 /usr/sbin/mysqld(+0xeb19e5)[0x55e2c505e9e5]
94 95 /usr/sbin/mysqld(_Z22recv_recover_page_funcmP11buf_block_t+0x9bd)[0x55e2c505fc2d]
96 /usr/sbin/mysqld(_Z20buf_page_io_completeP10buf_page_tb+0x51c)[0x55e2c51d3b7c]
97 98 /usr/sbin/mysqld(_Z12fil_aio_waitm+0x1b6)[0x55e2c524dfd6]
99 /usr/sbin/mysqld(io_handler_thread+0xc0)[0x55e2c5136c40]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7fe0842f44a4]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fe081fa5d0f]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.

Actions done 1 edited /etc/systemd/system/multi-user.target.wants/mysql.service added

LimitNOFILE=100000
LimitMEMLOCK=100000

systemctl daemon reload service mysql start

doesn't start

Actions done 2 edited /etc/mysql/percona-server.conf.d/mysqld.cnf added at the end

max_allowed_packet=1G
innodb_fast_shutdown=0
innodb_thread_concurrency=0

doesn't start

it seems a bug but a i have some important data to recover https://bugs.mysql.com/bug.php?id=91077

thanks

EDIT

root@XXXXXX /etc/default # ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 125599
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 125599
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
root@XXXXXX /etc/default # cat /etc/mysql/percona-server.conf.d/mysqld.cnf 
#
# The Percona Server 5.7 configuration file.
#
# One can use all long options that the program supports.
# Run program with --help to get a list of available options and with
# --print-defaults to see which it would actually understand and use.
#
# For explanations see
# http://dev.mysql.com/doc/mysql/en/server-system-variables.html

[mysqld]
user   = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket   = /var/run/mysqld/mysqld.sock
port   = 3306
basedir    = /usr
datadir    = /var/lib/mysql
tmpdir   = /tmp
lc-messages-dir  = /usr/share/mysql
explicit_defaults_for_timestamp

log-error    = /var/log/mysql/error.log

# Recommended in standard MySQL setup
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_ALL_TABLES

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

skip_name_resolve = ON
key_buffer_size = 300M
join_buffer_size = 300K

innodb_log_file_size    = 64MB
innodb_buffer_pool_size = 2GB
innodb-flush-method            = O_DIRECT
innodb-log-files-in-group      = 2
innodb-flush-log-at-trx-commit = 1
innodb-file-per-table          = 1

thread_cache_size=40  # from 8 to avoid thread starvation
innodb_lru_scan_depth=100  # from 1024 to reduce CPU busy every SECOND
key_cache_age_threshold=7200  # from 300 seconds to reduce key_reads RPS
key_cache_division_limit=50  # from 100 percent for HOT/WARM caches
key_cache_block_size=16K  # from 1K to evict bigger block when full

open_files_limit=10000  # from 1024 to reduce opened_files RPS
table_open_cache=10000  # from 407 to reduce opened_tables RPS
table_definition_cache=2000  # from 603 to reduce opened_table_definitions RPS
max_heap_table_size=48M  # from 32M  for additional capacity
tmp_table_size=48M  # from 32M to reduce created_tmp_disk_tables count
innodb_io_capacity=1600  # from 200 to allow more IOPS
read_rnd_buffer_size=192K  # from 256K to reduce handler_read_rnd_next RPS
sort_buffer_size=2M  # from 256K to reduce sort_merge_passes count
thread-cache-size = 50

# LOGGING #
# Setup log file locations for error log and slow log
# Slow log may be disabled on production setups to prevent extra IO
log-error                      = /var/log/mysql/mysql-error.log
log-queries-not-using-indexes  = 0
#slow-query-log                 = 1
#long_query_time                = 2
#slow-query-log-file            = /var/log/mysql/mysql-slow.log

EDIT 3

root@TSUNAMINUTRITION ~ # free -m
total        used        free      shared  buff/cache   available
Mem:          31421        1270       28759          76        1392       29644
Swap:             0           0           0

EDIT 4

2022-09-09T18:28:24.782352Z 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 30000)
2022-09-09T18:28:24.782483Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 10000)
2022-09-09T18:28:25.009689Z 0 [Warning] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release.
2022-09-09T18:28:25.009707Z 0 [Warning] 'NO_AUTO_CREATE_USER' sql mode was not set.
2022-09-09T18:28:25.012340Z 0 [Note] /usr/sbin/mysqld (mysqld 5.7.35-38) starting as process 29822 ...
2022-09-09T18:28:25.015518Z 0 [Note] InnoDB: PUNCH HOLE support available
2022-09-09T18:28:25.015545Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2022-09-09T18:28:25.015549Z 0 [Note] InnoDB: Uses event mutexes
2022-09-09T18:28:25.015552Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
2022-09-09T18:28:25.015554Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.8
2022-09-09T18:28:25.015557Z 0 [Note] InnoDB: Using Linux native AIO
2022-09-09T18:28:25.015928Z 0 [Note] InnoDB: Number of pools: 1
2022-09-09T18:28:25.016072Z 0 [Note] InnoDB: Using CPU crc32 instructions
2022-09-09T18:28:25.019210Z 0 [Note] InnoDB: Initializing buffer pool, total size = 2G, instances = 8, chunk size = 128M
2022-09-09T18:28:25.120045Z 0 [Note] InnoDB: Completed initialization of buffer pool
2022-09-09T18:28:25.165038Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2022-09-09T18:28:25.178502Z 0 [Note] InnoDB: Recovering partial pages from the parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite
2022-09-09T18:28:25.205015Z 0 [Note] InnoDB: Highest supported file format is Barracuda.
2022-09-09T18:28:25.216360Z 0 [Note] InnoDB: Log scan progressed past the checkpoint lsn 410343119828
2022-09-09T18:28:25.241184Z 0 [Note] InnoDB: Doing recovery: scanned up to log sequence number 410344579779
2022-09-09T18:28:25.241512Z 0 [Note] InnoDB: Database was not shutdown normally!
2022-09-09T18:28:25.241517Z 0 [Note] InnoDB: Starting crash recovery.
2022-09-09T18:28:25.286150Z 0 [Note] InnoDB: Created parallel doublewrite buffer at /var/lib/mysql/xb_doublewrite, size 31457280 bytes
2022-09-09T18:28:25.362340Z 0 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 18:28:25 UTC - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
Please help us make Percona Server better by reporting any
bugs at https://bugs.percona.com/

key_buffer_size=20971520
read_buffer_size=131072
max_used_connections=0
max_threads=152
thread_count=0
connection_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 351218 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.


Build ID: 33c8eb1b0ac2d6905fbf2e1e6c6c6a31a2976eb3
Server Version: 5.7.35-38 Percona Server (GPL), Release '38', Revision '3692a61'

Thread pointer: 0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0 thread_stack 0x40000
86 /usr/sbin/mysqld(my_print_stacktrace+0x2c)[0x563f2b23080c]
87 /usr/sbin/mysqld(handle_fatal_signal+0x3f6)[0x563f2b0607f6]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0)[0x7f0fa00dd0e0]
88 89 /usr/sbin/mysqld(_Z23page_cur_insert_rec_lowPhP12dict_index_tPKhPmP5mtr_t+0x360)[0x563f2b370c80]
90 91 /usr/sbin/mysqld(_Z25page_cur_parse_insert_recmPKhS0_P11buf_block_tP12dict_index_tP5mtr_t+0x660)[0x563f2b3728b0]
92 93 /usr/sbin/mysqld(+0xeb19e5)[0x563f2b34c9e5]
94 /usr/sbin/mysqld(_Z22recv_recover_page_funcmP11buf_block_t+0x9bd)[0x563f2b34dc2d]
95 96 /usr/sbin/mysqld(_Z20buf_page_io_completeP10buf_page_tb+0x51c)[0x563f2b4c1b7c]
97 98 /usr/sbin/mysqld(_Z12fil_aio_waitm+0x1b6)[0x563f2b53bfd6]
99 /usr/sbin/mysqld(io_handler_thread+0xc0)[0x563f2b424c40]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4)[0x7f0fa00d34a4]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f0f9dd84d0f]
You may download the Percona Server operations manual by visiting
http://www.percona.com/software/percona-server/. You may find information
in the manual which will help you identify the cause of the crash.
0 Answers
Related