Recommended log configuration in production

[edulid]

What is your recommended log configuration in production, so that it doesn't get gigabyte-big file per day but it contains information when errors happen?

I have the following, but today I got a sql error in production and I cannot find anything about the error in the logs.

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

	<appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
		<layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
		                  value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
		</layout>		
	</appender>

	<appender name="FILE_LOG" class="org.apache.log4j.FileAppender">
            <param name="File"   value="/smartClientServerLog.txt" />
            <param name="Append" value="false" />	    	
            <layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern" value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
            </layout>	    
	</appender>

    <!-- This appender is used by the "Server Logs" tab in the Developer Console. -->
	<appender name="SmartClientLog" class="com.isomorphic.log.RevolvingMemoryAppender">
	    <param name="maxEntries" value="500"/>
        <layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
		                  value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
		</layout>	
	</appender>

    <!-- isomorphic log messages -->
    <!-- The STDOUT appender defined above is set as the default appender for all Isomorphic
         SmartClient server logging.  It should typically be visible in the console from which
         you started the application server.
         
         Some Application Servers redirect stdout output to a file (typically called stdout.log
         or something like that, so you should be able to find it there.

         If you can't find the server log, you should be able to use the SmartClientLog (above)
         which is visible in the Developer Console under "Server Logs" and as a standalone log
         viewer in /tools/serverLogViewer.jsp.  
         
         Failing all of the above, you can switch the appender-ref value for the com.isomorphic
         category to use the FILE_LOG appender which will log to /smartClientServerLog.txt on
         UNIX and C:\smartClientServerLog.txt on Windows.  You can change this by modifying the
         value of the "File" param in the FILE_LOG definition above.
    -->
  	<category name="com.isomorphic">
	  <priority value="WARN" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>

        <category name="com.isomorphic.base.Reflection">
          <priority value="WARN" />
        </category>

        <category name="com.isomorphic.base.ISCInit">
          <priority value="WARN" />
        </category>

        <category name="com.isomorphic.base.ConfigLoader">
          <priority value="WARN" />
        </category>

	<category name="com.isomorphic.download.Download">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.sql.ValueSet">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.servlet.RequestContext">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.datasource.DataSource">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.datasource.BasicDataSource">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.validation.Validation">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.store.DataStructCache">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.assembly.FileAssembler">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.obfuscation.Obfuscator">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.servlet.PreCache">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.timing.Timing">
	  <priority value="WARN" />
	</category>
    
   	<category name="com.isomorphic.resultData.ResultData">
      <priority value="WARN" />
    </category>

    <category name="com.isomorphic.js.JSSyntaxScannerFilter">
        <priority value="WARN" />
    </category>

	<category name="com.isomorphic.interfaces.InterfaceProvider">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.compression.Compression">
	  <priority value="WARN" />
	</category>

	<category name="com.isomorphic.servlet.ProxyHttpServletResponse">
	  <priority value="WARN" />
	</category>

	<category name="org.apache.velocity">
	  <priority value="WARN" />
	</category>

	<category name="org.hibernate.SQL">
	  <priority value="WARN" />
	</category>

    <!-- shows values bound to params in SQL query, but produces a lot of output -->
    <!--
	<category name="org.hibernate.type">
	  <priority value="WARN" />
      <appender-ref ref="STDOUT"/>
	</category>
    -->

	<!-- embedded tomcat -->
	<category name="org.apache">
	  <priority value="WARN" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>

    <!-- hibernate log messages -->
  	<category name="org.hibernate">
	  <priority value="WARN" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>
</log4j:configuration>

[Isomorphic]

Generally we would recommend using the default configuration shipped with the SDK. That would show SQL errors.

[edulid]

Which one exactly? samples\showcase\war\WEB-INF\classes\log4j.isc.config.xml ? You got in your samples directory some different files. Or are they all equal?

[edulid]

This one is in samples\showcase\war\WEB-INF\classes\log4j.isc.config.xml. It contains INFO for almost everything. Should I use this one ?

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

	<appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
		<layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
		                  value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
		</layout>		
	</appender>

	<appender name="FILE_LOG" class="org.apache.log4j.FileAppender">
            <param name="File"   value="/smartClientServerLog.txt" />
            <param name="Append" value="false" />	    	
            <layout class="org.apache.log4j.PatternLayout">
		<param name="ConversionPattern" value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
            </layout>	    
	</appender>

    <!-- This appender is used by the "Server Logs" tab in the Developer Console. -->
	<appender name="SmartClientLog" class="com.isomorphic.log.RevolvingMemoryAppender">
	    <param name="maxEntries" value="500"/>
        <layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
		                  value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
		</layout>	
	</appender>

    <!-- isomorphic log messages -->
    <!-- The STDOUT appender defined above is set as the default appender for all Isomorphic
         SmartClient server logging.  It should typically be visible in the console from which
         you started the application server.
         
         Some Application Servers redirect stdout output to a file (typically called stdout.log
         or something like that, so you should be able to find it there.

         If you can't find the server log, you should be able to use the SmartClientLog (above)
         which is visible in the Developer Console under "Server Logs" and as a standalone log
         viewer in /tools/serverLogViewer.jsp.  
         
         Failing all of the above, you can switch the appender-ref value for the com.isomorphic
         category to use the FILE_LOG appender which will log to /smartClientServerLog.txt on
         UNIX and C:\smartClientServerLog.txt on Windows.  You can change this by modifying the
         value of the "File" param in the FILE_LOG definition above.
    -->
  	<category name="com.isomorphic">
	  <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>

        <category name="com.isomorphic.base.Reflection">
          <priority value="INFO" />
        </category>

        <category name="com.isomorphic.base.ISCInit">
          <priority value="INFO" />
        </category>

        <category name="com.isomorphic.base.ConfigLoader">
          <priority value="INFO" />
        </category>

	<category name="com.isomorphic.download.Download">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.sql.ValueSet">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.servlet.RequestContext">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.datasource.DataSource">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.datasource.BasicDataSource">
	  <priority value="INFO" />
	</category>

        <category name="com.isomorphic.datasource.AuditDSGenerator">
            <priority value="INFO" />
        </category>

	<category name="com.isomorphic.validation.Validation">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.store.DataStructCache">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.assembly.FileAssembler">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.obfuscation.Obfuscator">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.servlet.PreCache">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.timing.Timing">
	  <priority value="INFO" />
	</category>
    
   	<category name="com.isomorphic.resultData.ResultData">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.js.JSSyntaxScannerFilter">
        <priority value="INFO" />
    </category>

	<category name="com.isomorphic.interfaces.InterfaceProvider">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.compression.Compression">
	  <priority value="INFO" />
	</category>

	<category name="com.isomorphic.servlet.ProxyHttpServletResponse">
	  <priority value="INFO" />
	</category>

    <category name="com.isomorphic.naming.JNDI">
      <priority value="WARN" />
    </category>

	<category name="com.isomorphic.scripting">
	  <priority value="INFO" />
	</category>
	
	<category name="org.apache.velocity">
	  <priority value="WARN" />
	</category>

	<category name="org.hibernate.SQL">
	  <priority value="DEBUG" />
	</category>

    <!-- shows values bound to params in SQL query, but produces a lot of output -->
    <!--
	<category name="org.hibernate.type">
	  <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
	</category>
    -->

	<!-- embedded tomcat -->
	<category name="org.apache">
	  <priority value="INFO" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>

    <!-- hibernate log messages -->
  	<category name="org.hibernate">
	  <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
	</category>
</log4j:configuration>

[Isomorphic]

Any of the ones in sample projects other than the Showcase.

The Showcase is almost always the wrong place to take project configuration from. It is a very special project that needs to demonstrate every kind of persistence and related framework integration at once, totally unlike any normal project.

[edulid]

This is the file found in the built-in-ds:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">

<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

    <appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
        <param name="encoding" value="UTF-8"/>
        <layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
                          value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
        </layout>        
    </appender>

    <appender name="FILE_LOG" class="org.apache.log4j.FileAppender">
            <param name="encoding" value="UTF-8"/>
            <param name="File"   value="/smartClientServerLog.txt" />
            <param name="Append" value="false" />            
            <layout class="org.apache.log4j.PatternLayout">
        <param name="ConversionPattern" value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
            </layout>        
    </appender>

    <!-- This appender is used by the "Server Logs" tab in the Developer Console. -->
    <appender name="SmartClientLog" class="com.isomorphic.log.RevolvingMemoryAppender">
        <param name="encoding" value="UTF-8"/>
        <param name="maxEntries" value="1000"/>
        <layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern"
                          value="=== %d [%.4t] %-5p %c{1} - %m%n"/>
        </layout>    
    </appender>

    <!-- isomorphic log messages -->
    <!-- The STDOUT appender defined above is set as the default appender for all Isomorphic
         SmartClient server logging.  It should typically be visible in the console from which
         you started the application server.
         
         Some Application Servers redirect stdout output to a file (typically called stdout.log
         or something like that, so you should be able to find it there.

         If you can't find the server log, you should be able to use the SmartClientLog (above)
         which is visible in the Developer Console under "Server Logs" and as a standalone log
         viewer in /tools/serverLogViewer.jsp.  
         
         Failing all of the above, you can switch the appender-ref value for the com.isomorphic
         category to use the FILE_LOG appender which will log to /smartClientServerLog.txt on
         UNIX and C:\smartClientServerLog.txt on Windows.  You can change this by modifying the
         value of the "File" param in the FILE_LOG definition above.
    -->
      <category name="com.isomorphic">
      <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
    </category>

    <category name="com.isomorphic.base.Reflection">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.base.ISCInit">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.base.ConfigLoader">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.scripting">
      <priority value="WARN" />
    </category>

    <category name="com.isomorphic.download.Download">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.sql.ValueSet">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.servlet.RequestContext">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.servlet.IDACall">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.DataSource">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.BasicDataSource">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.DSRequest">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.DSResponse">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.PoolableDataSourceFactory">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.datasource.DataSourceManager">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.validation.Validation">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.store.DataStructCache">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.assembly.FileAssembler">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.obfuscation.Obfuscator">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.servlet.PreCache">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.timing.Timing">
      <priority value="INFO" />
    </category>
    
       <category name="com.isomorphic.resultData.ResultData">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.js.JSSyntaxScannerFilter">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.interfaces.InterfaceProvider">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.compression.Compression">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.servlet.ProxyHttpServletResponse">
      <priority value="INFO" />
    </category>

    <category name="com.isomorphic.naming.JNDI">
      <priority value="WARN" />
    </category>

    <category name="com.isomorphic.pool.ISCKeyedObjectPool">
      <priority value="INFO" />
    </category>

    <!-- Velocity - set at DEBUG to enable logging of resolution failures -->
    <category name="org.apache.Velocity">
      <priority value="DEBUG" />
    </category>

    <category name="org.hibernate.SQL">
      <priority value="DEBUG" />
    </category>

    <!-- shows values bound to params in SQL query, but produces a lot of output -->
    <!--
    <category name="org.hibernate.type">
      <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
    </category>
    -->

    <!-- embedded tomcat -->
    <category name="org.apache">
      <priority value="INFO" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
    </category>

    <!-- hibernate log messages -->
      <category name="org.hibernate">
      <priority value="DEBUG" />
      <appender-ref ref="STDOUT"/>
      <appender-ref ref="SmartClientLog"/>
    </category>
</log4j:configuration>

As you see here:

 <category name="org.hibernate.SQL">
      <priority value="DEBUG" />
    </category>

The SQL is configured to output DEBUG or above.
Is this the correct setting or may this setting be creating a lot of unnecessary logs?

[edulid]

I have a LOT of output of the form:

=== 2016-05-25 16:54:58,786 [ec-9] INFO  SQLDataSource - [builtinApplication.dokumente_fetch] Performing fetch operation with
    criteria: {criteria:[{criteria:[{value:3914,fieldName:"f_schueler_id",operator:"equals"},{criteria:[{criteria:[{value:15282,fieldName:"f_vertrag",operator:"equals"}],operator:"or"},{fieldName:"f_vertrag",operator:"isNull"}],operator:"or"}],operator:"and"},{value:1,fieldName:"f_mandant",operator:"equals"}],operator:"and",_constructor:"AdvancedCriteria"}    values: {criteria:[{criteria:[{value:3914,fieldName:"f_schueler_id",operator:"equals"},{criteria:[{criteria:[{value:15282,fieldName:"f_vertrag",operator:"equals"}],operator:"or"},{fieldName:"f_vertrag",operator:"isNull"}],operator:"or"}],operator:"and"},{value:1,fieldName:"f_mandant",operator:"equals"}],operator:"and",_constructor:"AdvancedCriteria"}
=== 2016-05-25 16:54:58,786 [ec-9] INFO  SQLDataSource - [builtinApplication.dokumente_fetch] derived query: SELECT $defaultSelectClause FROM
                t_dokumente

             WHERE $defaultWhereClause ORDER BY $defaultOrderClause
=== 2016-05-25 16:54:58,786 [ec-9] DEBUG SQLDataSource - [builtinApplication.dokumente_fetch] Executing row count query: SELECT COUNT(*) FROM
                t_dokumente

             WHERE $defaultWhereClause
=== 2016-05-25 16:54:58,786 [ec-9] DEBUG SQLDataSource - [builtinApplication.dokumente_fetch] Eval'd row count query: SELECT COUNT(*) FROM
                t_dokumente

                  WHERE (((t_dokumente.f_schueler_id = 3914 AND t_dokumente.f_schueler_id IS NOT NULL) AND (((t_dokumente.f_vertrag = 15282 AND t_dokumente.f_vertrag IS NOT NULL)) OR (t_dokumente.f_vertrag IS NULL))) AND (t_dokumente.f_mandant = 1 AND t_dokumente.f_mandant IS NOT NULL))

So for each SQL fetch I am getting 3 logs with this configuration. For development this may be ok, but for production I am getting gigabites of log files.
How to correct this?

[Isomorphic]

We don't have a specific recommendation for production log levels. It just depends upon how much or how little you want or need to capture.

All log categories can be turned up or down and the categories of each log are shown when they are output, so you can reduce the SQL logging if you wish, just as with any other category.

[edulid]

Where is the category you mention?
In this example: I see the level DEBUG on the class SQLDataSource. But the SQLDataSource is not the category, but the logging class. So how to turn this down?

 
 === 2016-05-25 16:54:58,786 [ec-9] DEBUG SQLDataSource - [builtinApplication.dokumente_fetch] Executing row count query: SELECT COUNT(*) FROM                 t_dokumente               WHERE $defaultWhereClause

[edulid]

Ok, the following seems to work:

<category name="com.isomorphic.sql">
      <priority value="INFO" />
    </category>

I still have some problems with the logging, but now regarding to missing logs. I will try to investigate more for this issue and will update here.