cannot parse as a valid JSON object, Oracle, Zabbix Proxy

cannot parse as a valid JSON object: invalid JSON object value starting character

Some other signs:

• active proxy
• zabbix proxy is also monitored by agent (127.0.0.1)
• there is a lot of active items on agent
• oracle XE 10.2 as tatabase engine
• a discovery rule has been scheduled behind proxy to scan local network
• zabbix sender sends the data to $(hostname). it is same hostname to monitor proxy health
• incorrect IP addresses for agents behind proxy. recheck all agent hosts behind proxy

There is a possibility that the problem comes from zabbix sender witch sends the data to the $(hostname). The same hostname which is configured for to monitor the proxy server health. Please confirm.

I did receive this on Oracle database hosting on proxy. The reasons was broken database because of huge item count and the database was reached 5GB limit. Solution was to drop the database and create new 20GB database.

# stop zabbix proxy
systemctl stop zabbix-proxy zabbix-agent

# make sure everything is down
ps aux|grep zabbix

# move to oracle user
su - oracle

# prepare sql script
cd
v=4.0.3
curl -L "http://downloads.sourceforge.net/project/zabbix/ZABBIX%20Latest%20Stable/$v/zabbix-$v.tar.gz" -o zabbix-$v.tar.gz
tar -vzxf zabbix-$v.tar.gz -C .
cd ~/zabbix-$v/database/oracle
cp *.sql ~

#see where the actual table space (fancy word for database in oracle termilogy) is hosting
find ~ -name *.dbf | xargs ls -lh 

# authorize into oracle as super user
sqlplus sys as sysdba
# enter password '5sRj4GXspvDKsBXW'

# drop zabbix table space
DROP TABLESPACE "ZABBIX_TS" INCLUDING CONTENTS AND DATAFILES;
# this will take 10 seconds or something

CREATE TABLESPACE "ZABBIX_TS" DATAFILE '/u01/app/oracle/oradata/XE/zabbix_ts.dbf' SIZE 100M AUTOEXTEND ON NEXT 536870912 MAXSIZE 20G LOGGING ONLINE PERMANENT BLOCKSIZE 8192 EXTENT MANAGEMENT LOCAL AUTOALLOCATE SEGMENT SPACE MANAGEMENT AUTO;
CREATE USER zabbix IDENTIFIED BY zabbix DEFAULT TABLESPACE ZABBIX_TS;
GRANT DBA TO zabbix WITH ADMIN OPTION;

exit

# authorize into zabbix database with password 'zabbix'
sqlplus zabbix/zabbix

# make sure the collation is right
select parameter,value from v$nls_parameters where parameter='NLS_CHARACTERSET' or parameter='NLS_NCHAR_CHARACTERSET';
# this should report:
# NLS_CHARACTERSET
# AL32UTF8
# 
# NLS_NCHAR_CHARACTERSET
# UTF8

# insert schema
SET DEFINE OFF 

@/u01/app/oracle/schema.sql

# exit oracle client
exit

# exit oracle user
exit

# clear proxy log
> /var/log/zabbix/zabbix_proxy.log

# start proxy
systemctl start zabbix-proxy zabbix-agent

# check logs
cat /var/log/zabbix/zabbix_proxy.log