Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Request] WithProperty Row Skipping, Casting Override, Error Handling, & Data sanitization #30

Open
dipique opened this issue Jun 9, 2021 · 0 comments

Comments

@dipique
Copy link

dipique commented Jun 9, 2021

Row Skipping

In the setPropertyCallbackValue callback, I can currently call Abort to stop processing. I would like to be to call Skip or Ignore to simply omit the current row. The ability to call a SkipProperty function would also be extremely useful; this would continue processing the row, but not attempt to convert the current property.

In my current project, I am currently achieving Skip functionality using a helper method like this:

        public List<T> Read<T>(string filename, string wsName, Func<ExcelWorksheet, ICollectionPropertyConfiguration<T>> txObjs, int startRow, Func<T, bool> valid = null) where T: class, new()
        {
            using (var xl = new ExcelPackage(new FileInfo(filename)))
            {
                var ws = xl.Workbook.Worksheets[wsName];

                var retVal = new List<T>();
                if (ws == null)
                    return retVal;

                var tgt_cnt = ws.Dimension.Rows - startRow + 1; //number of rows to read before complete
                var read_cnt = 0; // tracks # of read rows
                while (read_cnt < tgt_cnt)
                {
                    var vals = txObjs(ws).GetData(startRow + read_cnt, ws.Dimension.Rows);
                    read_cnt += vals.Count();
                    retVal.AddRange(valid == null ? vals : vals.Where(valid));                    
                }

                return retVal;
            }
        }

The txObjs parameter is a configuration broken out like this:

        public static ICollectionPropertyConfiguration<Person> PersonCfg(ExcelWorksheet ws)
            => ws.Extract<Person>()
                 .WithProperty(p => p.PersonId,  "A")
                 .WithProperty(p => p.LastName,  "B")
                 .WithProperty(p => p.FirstName, "C");

Casting Override

In one of the WithProperty callbacks (whichever makes more sense internally), I'd like to be able to call an OverrideCast method that allows me to write code to convert from object to the property type. This is for situations where there may need to be special logic for a specific property (like dirty data that sometimes contains a "0" instead of an empty string). This is particularly important because overriding the cast can prevent an error when casting an invalid value, such as an error.

Error Handling

Excel has a number of built-in error types. In one of the WithProperty callbacks (or as a separate callback), I'd like to handle the case of having an error value (e.g. "#N/A"). This could also be handled manually via the Casting Override change, but having an explicit error handling mechanism would provide a much more focused event to handle.

Data Sanitization

After the data has been converted to the destination type, I would like the ability to "sanitize" the data (or, if you prefer, transform it). For example, I may wish to convert "123456" to "R-123456", or trim strings. This probably makes sense as its own callback.


I think implementing these features would make this extension incredibly powerful for data ingestion and allow for very succinct and expressive code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant