Hello,
when schemaCheckEnabled is set to false we get confusing messages like this one (not exhaustive):
24/05/13 09:51:01 INFO BulkCopyUtils: Spark Dataframe and SQL Server table have differing column nullable configurations at column index 14 DF col XXX nullable config is true Table col XXX nullable config is true
in the class BulkCopyUtils, we have this implementation:
private def assertIfCheckEnabled(
cond: Boolean, checkEnabled : Boolean, msg: String): Unit = {
if(checkEnabled) {
assertCondition(cond, msg)
}
else{
logInfo(msg)
}
}
It's wrong as it displays misleading messages (saying there are differences) when the condition is true. So I suggest this patch:
private def assertIfCheckEnabled(
cond: Boolean, checkEnabled : Boolean, msg: String): Unit = {
if(checkEnabled) {
assertCondition(cond, msg)
}
else if (!cond) {
logWarn(msg)
}
}
Hello,
when schemaCheckEnabled is set to false we get confusing messages like this one (not exhaustive):
in the class BulkCopyUtils, we have this implementation:
It's wrong as it displays misleading messages (saying there are differences) when the condition is true. So I suggest this patch: