hive server is too talkative

During usual "production" use case, hive server shouldn't be that talkative as it generates huge amounts of mostly useless logs, not to mention that such extensive logging tends to have non-negligible impact on performance. Make such verbosity optional.

added To Do label

Recovered some notes (after the metadata oblivion incident ;-) )

@ABW

Problem with logs is that they are generally useless until something starts to break.

No example was provided and I only have log from 5M test sync, but I assume the issue is with timing reports. I guess we could skip such report if overall block time was below certain value (what threshold would be ok for such filter?) but if we aimed at filter based on deviation from rolling average, that could also have impact on performance.

Ok, so the options here we have are the following:

• split log between files every x-time - then it should be easier to remove old logs manually, or even automate the process and remove them when their number or cumulative size is too big
• we should generally not log responses at all - in case of a problem we can either reproduce it, and then response in log gives us nothing, or we cannot reproduce it, and then we have to try to reproduce it, meaning all the response in log gives us is the confirmation that something went wrong
• it doesn't show in the above example, but when both stdout and stderr are sent to log, some errors are spamming full callstacks - that is only needed when we crash
• we can shorten the prefix, f.e. INFO:jsonrpcserver.dispatcher.request: could become > ; it would be nice to be able to replace long error/warning prefixes with simple E/W (as long as we make sure to have easy way of finding the source of such message)

If we needed even less in log we could replace requests with statistics: f.e. how many times bridge.get_profile was called in last hour. However we should still show full requests in case they are particularily slow, which means we'd need a way to measure response time and track "typical" call length.

changed milestone to %JulyHivemind

Most of the logging was from stack dumps on exceptions reporting errors in api parameters. The primary culprits were fixed in !688 (merged) and !690 (merged)

closed