Handling ALL That Data

Handling ALL That Data

Several of my recent blog articles have been on data collection and use. Many readers have responded with the same questions. With data coming at me like a fire hose, how do I safely store and retrieve it? Being married to a professional landscape photographer, I learned years ago about putting large amounts of data in a safe place for future use. One advantage we have is that all our data may not need to be saved forever and a post job purge can reduce overall usage.

I will talk about different data storage methods and their advantages. I will not name companies. A search of product and service types will give you a wide range of options. When working on a project, some of the files will be kept on your local machine. To be proficient after some operation(s) are performed, you can move files to a remote location to stop your computer from slowing down.

The Desktop

Do what you need when a project is ongoing. In some cases, we dedicate a hard drive to a project and then store it after the project is completed. Hard drives can range from 100 megabytes to over 10 terabytes. Disk size for hard disk drives (HDD) can be over 20 terabytes at a reasonable price. Here are the best practices for desk side storage.

  • Have a large (2TB+) solid state drive (SSD) for Windows and other programs.
  • Have a second solid state drive for file storage. We have added multiple SSD’s for expanded storage. Prices are low and performance is required for data modeling and photogrammetry.
  • Keep things on the drive as the project goes along. Remote storage is good, but a lot of files may need to be brought back to the SSD’s to do some more work.

I have not mentioned backup of your local machine yet. We will cover local (somewhere you can get to easily) and cloud backups. Remote backup is also another option.

Local Backup

The easiest way to make sure things are safe is to back up your hard drive and put it in a fire safe. A process that is time consuming and prone to forgetfulness. I do not recommend this as there are other solutions.

Network Attached Storage (NAS)

NAS is the easiest way to grab something, work on it, then put it away safely. As the name implies, you have a large amount of storage that is connected to your network. It is a drive that is labeled as a letter, I use “N” for simplicity. The drive itself acts as a small computer with a disk array for redundancy in case of partial failure. Transfer speeds are generally good making it easy to retrieve and save data. There are other benefits as well.

Another NAS drive can be placed offsite, giving you added protection in the event of complete destruction of one of the locations or its drive. I did this for our firm years ago and had a host of issues getting things set up. There are a number of independent consultants who will do this in a matter of minutes and it is well worth the money. Here is a road map.

  • Find a technician to set up a NAS. The brand you buy will depend on what the technician recommends.
  • Decide on the amount of storage you want. The technician can help with that.
  • You will get two NAS drives. One will go in the office, the other in a remote location that has a fast internet connection.
  • Both drives will start out in the office, the initial large data transfer will be easier that way.
  • The drives are synced and tested while in one location, this makes troubleshooting easier.
  • The second drive goes to the remote location.
  • Drives can be set to back up continuously or at a specified time, usually early morning hours when no bandwidth is being consumed

This peace of mind is critical in securing your data information. Also make sure that important accounting data and correspondence gets saved from other staffers and departments in the office. The cost for this is less than you may think, and you can avoid sad stories of lost data and ransomware.

Cloud Backup

Whenever I mention the possibility of using cloud backup, I often get a response related to security. First, platforms like Dropbox are secure and second, the plans and files you are saving are usually public domain if somebody wants to look them up. How tragic is it if somebody sees the CAD for a subdivision anyway? The exception I will agree is sensitive correspondence, financial information, and most internal communication. Use the NAS for that and trust the rest online. Here is the process.

  • Decide on the platform and storage limits. Prices can vary widely so shop around. Security is similar with most so do not let that be an issue.
  • Decide on what to sync. I use the sync folder as local access as well because the data resides locally and is mirrored to the cloud.
  • Depending on your connection to the internet, you can choose to continually update or pick a specified time. This is usually in the early morning hours when you are hopefully not working, and speeds are good.
  • A dashboard for the program will keep you posted with the status of a sync.

This easy solution may be the best option due to the protracted setup of a NAS. If you have followed along to this point, there is still the possibility of accounting and correspondence not being saved. Do not delay the install of a NAS too long.

Purging Stored Data

We never really get rid of information completely. After a time, we save a distillation of information and lock it away. You may also want to check with your attorney as to how long they feel you should have full saves as opposed to the reduced density mentioned. Here are some examples.

  • Regarding machine control data, we save the latest file we made in the native software and the last files sent to the field. The pdf’s can be saved as well. Sometimes if they are huge, we will strip out pages outside our scope. Remember they are generally available at the agency responsible for approvals. Let them warehouse it. We do not need the original CAD files.
  • Photogrammetry jobs get saved as the point cloud and GEO-tiff. Raw images and initial surface production can go away. With the point cloud we can generate what is necessary in case we need to go back. That will save a ton of space.
  • Takeoffs can usually be the program file we used for the numbers. No reports need to be saved as you can run them again. We save the pdf sheets associated with the takeoff as well.
  • Job notes and correspondence can get saved in total. They are almost always copies of emails and letters that take up little room. I have spent more time than it is worth trying to decide what should stay and the space saved is minimal.

Well, there you have it. A comprehensive plan for saving and securing data. The pandemic has changed everything and many of you are working at home and need access like as if you were in your office. When I started this company, we all worked remote and were on the cutting (painful) edge of a lot of the previously mentioned technologies. Things have gotten much simpler and integrating some or all these ideas will give you a piece of mind. There are a lot of horror stories out there regarding lost or stolen data. Hopefully, you will not be one of them.

Civil Construction Amid COVID-19

Civil Construction Amid COVID-19

Disclaimer: This article does not contain health and safety protocols.

For years we talked about remote jobsite access where we could check a project’s progress by viewing live feeds from cameras and machine sensors. The mining industry has embraced this and my experience in this area will guide my thoughts and recommendations in this offering.

I will go through the COVID-19 workflow and outline some areas that you may want to invest in. These improvements will not be abandoned when the crisis passes but instead become necessary upgrades. When upgrading machines and software there might be components for the machine that need to be purchased. With the machinery improvements, the addition of software is required but not as cost intensive as upgrading machines.

Pre-Job Takeoff

Yes, I am going to mention the need for a good topo which results in a good takeoff. Drone flights can be done safely with results returned to the office. What about a site inspection? You’ll want to get a look at the job and get a feel for it. That is the art of estimating – we leave the good dirt numbers we create using software and look at what it will take to make things happen.

Drone Flight Basics

  • Get the crew out and set control points for an aerial topo.
  • Fly with a drone to get nadir imagery for conversion to a point cloud.
  • Verify the quality of the data in the field.
  • Reconfigure the drone to a gimbaled camera forward and down view. This gives an angle to see elevated items visually and not rely on the point cloud.
  • Fly two different patterns.
    • Run the “lawnmower strips” but lower and with the camera at the described angle.
    • Free fly to get specific areas that may need closer scrutiny. Go all around stands of trees, old buildings, and stockpiles as you would do on a site visit.
  • Process the drone point cloud as usual.
  • Using the ortho image of the job, make notes regarding the detail flights such as where they started and ended. A dotted line of a special flight will help the user get oriented quicker.
  • Rename the special flights and correspond the name to the legend on the map.
  • Have someone review the files you submit without explanation. After a few jobs you will be able to present a takeoff topo to the estimator that needs no further explanation.

Bidding

Not to go off into the weeds regarding proposals, but these times have created some special demands. Many of our clients are putting in disclaimers and questions in the bid for further clarification. We have even seen “if/then” line items so the owner knows what to expect if the pre job images could not tell the entire story.

Once the job has been won, it’s time to go to the field. Here are some ideas to consider.

The job trailer may be a thing of the past. Nobody can really go in so all it becomes is storage. Consider a smaller windowless container instead. With site visits being reduced, we need a way to communicate and keep everyone in the loop.

Job Meetings

We have had to make a quick shift from in person meetings to all remote interaction. This brings up a host of new challenges.

File Access

People need to see what is being done. Set up shareable folders online to give the right people access. Here is how this changes what gets shared.

  • Limit the ability of users to post files to your collection. Too many people adding documents can make a mess.
  • Use a separate folder for input from those outside your work group. Review the added file and keep it there or put it in the main slipstream for others to review and comment.
  • Turn off update notices. Doing this at the start will disable sub-folders as well. When something big has changed, let those that will be affected know.
  • Get file structure sorted before the project begins. You can waste hours looking for something in a rat’s nest of file trees and sub-folders. Yes, there will be a lot of files. They are usually updating to base versions. There can be a lot of folders with new and old files available to review. Maintain the dates for these folders but keep the structure and hierarchy.

Meeting Basics

Now that we are all virtual, there are some things that must change for things to keep moving smoothly.

  • Learn how to use your conferencing platform. Know how to mute, operate your video, and change backgrounds.
  • Learn how to get your face to look correct. Take a moment to review your video feed and learn what light looks good.
  • When using a cell signal for conferencing, things can get slow and you may miss a lot of dialog. Turn off your video to improve audio reception.
  • Have ONE person run the meeting. Weekly meetings should have the same person and agenda; yes it will change but keep the structure consistent.
  • The worst thing with remote meetings is everybody talking at once. To get over this, we have used raise hands, chat in a speak request or question, and going around the screen for comment or pass.
  • Virtual meetings do not have the same impact of in-person. After every meeting, somebody needs to distribute notes as to who said what and what happened so it can be reviewed and commented on.
  • Never have a meeting that could have been an email. How do you avoid this? Send out the item(s) as an email first. If there is too much comment or lack of agreement, then it gets to go to a meeting.

Change Orders

Simple change orders are no longer simple. Usually there is a site meeting, and the problem is hashed out. We have helped clients with this issue and there are steps to take that will help reduce the hassle. Start in the field, then move the information to the office.

Field Work

  • Just like with the takeoff, images will be the key to stating your case.
  • Drone and ground video with narration done in the office are worth a million words.
  • Collect topo data if needed.
  • Take a narrated cell phone video to explain the details to the office so the presentation is clear.

Office Work

  • With field data in hand, start to build a story that walks through the problem and proposed changes.
  • Always propose an answer.
  • Go through the data provided by the field and create the story. Bullet the high points to help with the steps of toe issue and a fix.
  • Talk to the field people in an online meeting to verify you have the details right.
  • Have the field collect any additional information needed to clarify things.
  • Now is the time to make the short, clear story for a person who has not been on the site to feel like they are.
  • Images and video with narration as well as the text of the narration included as a Word document are key.
  • Send the information back to the field and have them review. If it makes sense to them, you are good to go.
  • Do not schedule a meeting! Send the data to the parties involved for them to review and if there are too many questions or issues, then go to a meeting.
  • Use their questions and concerns to improve your template for subsequent presentations.
  • After a few of these, you will be quick and concise.

Summary

Nothing that I have mentioned will die with COVID-19. When we come through this, the ease of performing these ideas that I’ve outlined will continue to live on.

Upgrading machines to generate topo data as well as dirt movement is something worth looking into. I will cover this in a future offering.

Do not be concerned about the software expense and additional training. As you may know, we conduct our business remotely and have been employing these tools for years because we do not live near any of our jobs. From a quick markup of plans on Bluebeam, or a complete presentation with information provided by our client, we make difficult subjects clear.

Start doing this with small items to get your list in order. On larger issues, follow the same playbook and watch how easily and quickly things come together.

Processing Photogrammetry Images: Outsource or Inhouse?

Processing Photogrammetry Images: Outsource or Inhouse?

Many of our customers find that capturing images during the life of a project is useful. In this article, I will discuss the options available for processing images.

There are two options for image processing and surface generation – do it yourself or pay someone else. I will go over the good and bad associated with both options as well as a brief review of some platforms available.

Inhouse Processing

When I started out in this industry, we did all of our own image processing. There were no options. My advantage was having a wife who is a professional photographer. She gave me guidelines for manual camera settings to improve image quality. Our results were nothing short of amazing. Once I knew how to set the camera, we experimented with different flight patterns and coverage to dial that in.

The survey aspects of getting good control, setting and shooting ground control, and adding check shots is procedural.

Tips While Shooting in Field:

  • Control needs to be established, regardless of who processes the data.
  • Make sure the scale factor is between 1.0000 and .9999 for accurate data.
  • There is no established rule for the number of GCP’s (Ground Control Points) set. In flat areas, I may set one (1) per acre and hilly sites two (2) or three (3).
  • While walking from one control point to another, take topo shots for use later.
  • When your partner is flying a drone, walk around and shoot more topo points.
  • Make sure to occupy the GCP’s for enough time. Accuracy is key.
  • Topo shots can be quick.
  • Use the best camera available for your platform. Make sure the lens is clean.
  • Be sure to check the images in the field before you leave.
  • Time of day and cloud cover have a big impact on image quality. Sometimes you will need to wait for better conditions.

Overlap and flight lines are a bit of an art. Start with recommended numbers from the software provider. For example, Pix4D has different overlap specs for general dirt operations, dense vegetation, and corridors. Do not just run out and do 90% frontal and 90% side overlap “just to be sure.” You will waste time and processing will not be better than lower numbers.

Back at the office, work is a balance of hurry up then wait.

Steps to Take in Office:

  • Transfer images from the camera memory cards to your computer.
  • Batch process images if necessary (more on this later).
  • Import images into your software, as well as GCP’s.
  • Mark ground control on imported images.
  • Processing frequency varies by vendor. Assume 2 or 3 long processing waits while you do something else on another computer.
  • Invest in a high-quality standalone computer for photogrammetry. You will not be sorry. Your vendor will have recommended components. Do not skimp on any of them.
  • Go through the required steps to get a surface.
  • Review and verify the surface. The person(s) who flew the site should be there for review of the model. They can help with initial review and help to define out of tolerance areas.
  • Bring in your topo points and compare them to the surface. Confirm tolerance.
  • Filter the surface density to get what you need for comparison.

Surface Size

When the data processing is complete, I will want to keep the surface large. The reason is I like the way takeoff data software allows me to make a hybrid surface to get detail with a smaller file size. In most cases, a 10 or 20-foot grid will work. Carlson has a feature that allows a point to be added at a horizontal distance and/or a vertical change in order to get tops and toes of slopes as well as small channels or berms that get missed in a grid. Either way, there is no need to use a huge file size where a smaller one does the same thing. I have done testing. Try for yourself and you will find the balance as well.

When exporting a surface to be sent to others, here are a couple tips.

  • Send them the same surface density you are using.
  • An XML TIN is fine. Older software can use that TIN file saved as DXF triangles.

Batch Processing

This may seem like over-information, but I need to mention these details as it has saved us on many occasions. Adobe’s Lightroom can batch process images to make edits to all of them at the same time. In other words, if something is wrong with the images you captured, you may be able to be correct them using the program.

  • Underexposure due to cloudy conditions is the toughest. It is hard to adjust lighting from dark to light, however I have been able to improve flights by processing.
  • When in doubt, overexpose your images. It is easier to bring down contrast.
  • Sharpening can help, but usually causes problems. Best to get good sharpness from initial camera settings then try to adjust afterward.

The biggest issue is the fact that this represents another software to learn and a steep learning curve. As a professional who does a lot of this, it is just another software expense that saves us or a client another long trip to a site that may have changed drastically since we were last out there. In cases where the site is remote and you got there on a bad day, this option can make things work. Something to keep in mind if you start doing a lot of photogrammetry.

Using a Service Provider

The easiest way to get a surface from a flight is to upload the images to an online processing provider. There are several things to be aware of when doing this.

  • Everything gets processed the same. Be certain you know the exact overlap and image quality the provider expects.
  • They were not on the flight. Special instructions or interaction are not available. This prevents them from getting firsthand knowledge of the site to improve quality.
  • Results vary. Sometimes the same set of images sent to the provider at different times yield different results.
  • Turnaround time is going to vary.
  • Some services do not allow you to download a surface. You can only use their online tools.

These are not a lot of negatives to the process. I like using a third party especially for interim topos of dirt progress. Here are more advantages:

  • Click it and forget it. It is easy to get the images uploaded for results to come back.
  • Surfaces are stored online reducing the need for additional storage space in your system.
  • The online tools are great. Two useful tools include: sliders to see date changes to a surface and the ability to share with your client.
  • These services are mature, and results should be consistent. As a test, you can rename a group of images and compare surfaces.
  • You will fly your drone more if you know that someone else is doing the processing.

There are a lot of online services available. Both Trimble and Carlson have their own, with a lot of independents vying for your business. Here are some tips for selecting one:

  • Do not be tied to the service offered by your software. Look at it but consider others. The interface is usually a macro that automates some easy steps.
  • Prices vary widely. You will find everything from pay per megabyte to unlimited. Be sure you know what you are getting before committing.
  • Image size and overlap may be an issue so check their requirements. If they want a large overlap and you are paying for file size this can skew pricing significantly.
  • It is difficult for a provider to promise turnaround times. A trial can help you to see how efficient they are.
  • Online tools are important. Look for the ones that suit you best. Some sites are geared to mines, others to corridors. The surface and calculation features differ. Pick one that works for you.

Combining Methods

I have found the best results are when I can process in-house and then send data to a good provider. I can take care of the parts that need special attention and basic dirt numbers can go outside.

You need to be careful here. Never mix the service and your work without a baseline. Here is how to do this:

  • Gather the images. If your settings are different from the provider, you will need to make two flights.
  • Process your surface and send the images out for a surface.
  • Anytime you fly and process, only compare to your other work.
  • Never do a quantity verification mixing the two.
  • If you must mix, do a report of the recent outsourced surface to your new flight. Send the images out as well. Do a volume calculation and note the differences to adjust any subsequent work to be done on them.

 

3D Data on a Fast Track Civil Site

3D Data on a Fast Track Civil Site

The urgency to start projects and complete them quickly has become common. No matter what the use, the quicker the job is completed, the faster it’s used. Being first on the site, civil contractors generally have the least complete plans to start with. Owners do not understand that we need to know what the project is going to look like before we start planning. I have set some baselines for producing data for fast tracked projects that makes things easier for everyone.

There are three models that we will end up building which represent three distinct phases. Here are descriptions of the three basic model types for clarification.

  • Mass Excavation: This represents the bulk cuts and fills on the project. Expect no better than 1-foot accuracy.
  • Mass Grading: This brings things within a tenth. Retention and landscape areas should be good, and the building pads and parking areas may not be finalized.
  • Fine Grading: This is the final detailed information. Entries, parking, and building pads are all finalized as well as specifics for landscape, walks, and installations on site such as benches and playground equipment.

It seems no matter how much information we get, we always need more. I will go over the information we need and how to build each of the three models. Communication with all parties is critical. The example used is for an apartment complex. The owner is not sure how many units to build. Pre-leasing will help in that decision. In the meantime, pads, parking, and common areas are not finalized. We will work with engineers to make sure the contractor only moves dirt once and the project keeps running.

Mass Excavation

A “complete” set of plans needs to be submitted for approval. After permitting, things always seem to change. In this case, we will get enough information together to have scrapers and dozers working.

Before the crew gets to localize the site, equipment is moved in. We provided two different PDFs for use in the field. This may seem like something to keep the owner happy, but I want the work to count. The PDFs contain a 50-foot grid for cuts and fills. It’s enough for a couple days until the GPS can be set up on site and the real work begins.

As you can see in the image, we had finished contours for the retention to the east and rough grades for future pads. The bulk of the work is the buildings that are in the central area.

The plan set shows buildings and related access. With changes coming, we will smooth out the area and make some haul lanes. This is not a big site so we either need to start detailing or the contractor will have to leave, something that the owner does not want.

Even if the buildings change, we can get close to finish because there is usually not much elevation change between the pad and the paving areas. The biggest concern is not to build pads that are too big. This job required special compacted fill for the pads and the cost of an oversize pad would be substantial.

Summary

We can generally use the contours to get started. With larger cuts and fills, the time to get to the next phase is longer but the urgency is still there. Too much dirt moved or placed is not acceptable. Here are some tips:

  • Get a good OG topo. LIDAR or a drone topo is the way to go. Do not trust the contours from the plans if possible.
  • Be sure to get major drainage in at this point. No sense in making temporary ditches to move water.
  • If roads are being built, consider using that path for haul roads. Final grades can be cut on this well compacted surface easier than loose material.

Mass Grading

When we build this model, we know general footprints and elevations. I want to get things as close as a tenth of a foot. To do that, a lot of things need to be in place.

  • Pad sizes and elevations.
  • Streets should be close to finish. If the pads are good, street elevations are usually related to their elevations.
  • Utility rims and grates are set to finished elevations.
  • Parking lots are not detailed but close enough to rough things in.

For this job, I set the building pads to elevation and size. They will be handled as their own sub-set due to select fill and compaction requirements. The curb is laid out and we have a good idea of the 2D location. Final elevations will be worked out in the last file.

As you see her,  the design looks complete. It will allow the contractor to get the pads done and rough in the streets after the utilities are placed. I like to square up the pads, as that is the way they will be built. In this case they requested the actual layout.

When you are doing a job like this with a lot of detail in a small place, it is important to manage material. There is not a lot of room for excess dirt and we need to be careful with frequent drone topos to get rid of just enough material.

Summary

In a perfect world we could reduce files to no more than two. That is often the case if the rough grade file is ready to go and in a week we have the final information needed to make the fine grading surface. With the job described, the contractor needed plans to get them closer to finish without the final plans being ready. The rough grade file can buy the office some time to get the fine grading file ready. It’s important to walk away from a job and look at it with fresh eyes in the morning to get the last breaklines and spots for better performance.

Fine Grading

With one more opportunity to get questions answered, we now create the fine grading surface. Here is what we are going to produce for this surface.

  • Retention areas to the correct volume required for the changes in the project along the way.
  • Correct slopes away from buildings for drainage.
  • Streets that drain to properly sized inlets.
  • Intersections that work with traffic and shed water. No humps in main streets that will not work with posted speeds and pullouts and deceleration lanes that drain.
  • Sidewalk ramps to ADA standard with sidewalk cross slopes at less than 2%.
  • Parking lots graded to move water and contain no abrupt slope changes to catch low front scoops.
  • As many breaklines and additional spot elevations required to make all the above work.

As you can see from the image, we added a lot of information to make the final file. These details will make the model perform better as well as give the contractor a look at what will be required to finish the project.

The green lines in this model are breaklines done by our engineer Michael Stallings, to help the surface do what we want. These are 3D lines that connect elevations on the surface to make sure the TIN links with correct elevation points. Of all the things in a model, the breaklines are the most confusing for new model builders. There is no easy answer. You need to draw a breakline in an area the does not look right to see if it improves the performance. If it does not, delete and try something else.

Summary

You want the fine grading file to be sent out last. We have had jobs where we sent out ten files as revisions and updates. The fine grading file should be a finalized surface of the information you have. If you do not have that, do not call it fine grading. The field needs to understand what the different file types do and know the tolerances placed on them.

Conclusion

Many times, we can get one file out the door with all the information needed built at once. We only hear from the contractor for the next job. I hope you have the same result and do not need to send multiple files. The following process will save you time and trouble.

  • Keep file names consistent. If you use “Mass Grading” use it every time. The field will know it is close but not final, they can work with that in mind.
  • Put dates on every file. It saves confusion.
  • No need to bring a lot of equipment to the site initially. Start with some non-GPS machines for initial work and bring in guided equipment as things get moving.
Carlson and Civil 3D: The Details

Carlson and Civil 3D: The Details

Let’s get to the meat of the program and see how data is produced. Before sending things out to the field, we need to make 2D and 3D lines as well as points. As I go through the steps, you will see some different approaches to the process that may help users with various situations.

Getting Around

You will need to get some things out of the way to create data. Starting with Drawing Cleanup is where we’ll begin. There are a lot of options for cleaning including bringing in CAD elements that may otherwise get lost in other platforms.


The following settings will work fine for this drawing. When completed I can get the layers stripped and create a layer state for building data.

 

 

 

 

 

 

Listed below are notes on Drawing Cleanup results:
• I got a lot of help here with lines and verticies. I will add line points back later, but at this stage I want a straight line to have no interruptions until I add them.
• The duplicate lines were extreme in this file. Looks like it came from a couple xref plan lines that were stacked on each other when they exported everything to one file.
• The conversion of different linetypes into polylines will make things easier. Sometimes when you are elevating along curb, the line will break because it is next to a different type. Now I can join and elevate easier. This process is critical not only for data, but the field controllers will either not display or even crash with other than polyline exports.

With the unnecessary layers turned off, we are now ready to produce a file to send to the field.

The Commands

Let’s review some ways to produce data components. The first will be 2D lines (contours) that will need some work later, but I want to see what we have to start with. Contours are usually not a big deal. The advantage of working in AutoCAD is the ease of changing and editing polylines to make the contours work well.
• It is easy to convert any line type to a polyline.
• Direction can be reversed to help joining lines.
• Individual lines or layers can reduce or densify vertices.
• Points at intersections can be added.
• Move the start of a line.
• Add or remove arcs from polylines. An arc cannot be elevated so segments must be added for elevations to be added.
• Highlight non-tangent, non-perpendicular, crossing, and unclosed polylines. These tools are great for getting to the bottom of problems with detail areas in a model.

Elevating polylines can be done with the command shown. The properties can be changed as well so you know what has been elevated.
• Set the interval to match.
• Lines can be re-layered if desired.
• Color can be changed.
• Line type and the width of the lines can be changed.
A tip I like to use is to re-layer the location of the elevated polylines and make that layer un-selectable in case you pick the line again so it does not get changed. While in the process of making contours, it is a good time to check and see if there are any additional line fragments under or maybe copied to the new layer. Housekeeping at this point is critical.

3D Lines

Elevating curb lines is the most tedious and potentially problematic part of civil site data. We can go through the plans and pay attention to called out elevations to correctly enter them into the right vertex. Only after the model is built and the line is offset for back of curb do we sometimes notice an issue.
Carlson has helped with a visual representation of a polyline as you add elevations to spot issues before going too far.

In this image we see the polyline edit functions. I will outline this in the video but it is worth drilling down here.
• The profile box shows the elevation of the selected node.
• If I add a checkbox, it will hold that elevation.
• Incoming and outgoing slopes are shown.
• I can add vertices by double clicking anywhere on the line.
• I can move a vertex to adjust line location.
• Vertices can be added by crossing polylines, useful for elevation callouts that have a leader intercepting the subject line.
• If I have a surface, any location on the line can be elevated by the surface.
The process of going through a line and elevating can go quickly. Using the profile will expose fat-fingerings as well as incorrect callouts.

Curb Offsets

I use the Offset 3D Polyline command for curbs. I will discuss that in a moment but wanted to go over the dialog box because it contains a lot of information.
• The Interval method is for one horizontal and one vertical offset. This is good for simple curb offsets usually a half a foot in each direction.
• The Variable method allows you to change the settings per line.
• We will cover the Multiple method in more detail later.
• Constant method will perform a horizontal offset and make the line into a single elevation. Think building pad offset or basement excavation.
• Surface method lets you make a defined slope into an existing surface for a daylight line.
• Intersection method is for making a 3D line at the potential intersection of various slopes and two reference polylines.
• The layer of the newly generated line can be placed on a new layer.

The Multiple command is a strong tool. Here are some details.
• Progressive offsets makes the next change to the last line entered. Unchecked and all changes are made to the reference line.
• There are three slope options available. They are Vertical offset (in feet), Percent (can be positive or negative), and Ratio as the entered number x:1
• While in the command I can make a layer name without going back to the layer manager. The layer color will be white with a solid line but that is easy to change.
• A click on the three dots that are to the far right, will bring up a layer list where I can pick the correct previously entered layer. Any difference to an existing layer entry from spelling or spacing will create a new layer.

With CAD standards it is easy to pick the layer name from the dropdown menu when things are set up well. Always bring a drawing into your established template.

Improving a surface

When the basics are out of the way, you will need to clean up areas that are not performing well. Adding breaklines in Carlson is not as easy as other programs mainly because the surface needs to be remade to see the changes. Not a big deal but real time work is not possible. In this case, I would use tight tolerance contour lines (.1 feet) to see results in plan view before going to 3D view for confirmation.

The process is not difficult, and it is made easier when using SiteNet because layers do not need to be turned off for the surface to be made. I’ll go over this in the video.

When adding additional 3D information to a job there is a slippery slope. Less is more but how do you know? Here are some guidelines.
• First make certain that you need to add not take away. In other words, can you trim a line a bit or delete something for the surface to perform better?
• Always put breaklines on their own layer. Sometimes I won’t include the breaklines in a surface to see if I did too much.
• This is the most time intensive part of a project. To become proficient practice on small sites. Large jobs can have hundreds of “improvements” and many of which are not needed.

Making Surfaces

It is easy to make a surface for a software’s field computers. Carlson has field software but what happens when we build for another brand? We have the best luck with xml surfaces and either dwg or dxf for linework.
Surfaces are easy to make, and there are plenty of options.
• The tabs let you control contours and labeling. You are also able to choose the data types to be used for that surface.
• TIN lines and/or faces can be drawn.
• Slope labels can be added to the screen.
• Boundaries can be drawn in CAD and selected. Carlson can also shrink wrap the surface elements.
• You can model a cliff by changing the reference plane.
• There are some great tools for existing surface production. These tools are useful when you have a big group of contours.

Exporting Data

The following outline is our practice for exporting surfaces.
• We make Carlson TIN files. It is native to the program and is needed for many of the tools.
• We verify the linework is polylines and joined where needed. This is for 2D lines only and that work was done for converting things to 3D, so they do not need to be reviewed.
• We will use a dwg export if the field software is able to convert it, dxf is the second choice and is usually a larger file.
• Xml export is easy to work with, just be sure you have the correct units.

Carlson Software: Under the Hood

Carlson Software: Under the Hood

There are a lot of things to like about a software program. In the case of Carlson, I think it is the blend of old and new. Carlson software is mature and there are commands I am still finding that I either forgot about or never used. I am sure that some commands have been dropped over the years, but users have access to all manner of niche ways to manipulate data.

The Basics

Carlson runs on top of a CAD program, either Autodesk or on free IntelliCad. For many users, the free CAD program is fine. We need power to do things with data and for the money Carlson Civil on Autodesk Civil 3D is the way to go. This gives you the power of Carlson commands and the native software of over 95% of our civil site deliverables.

Anyone who has dealt with Civil 3D files knows the frustration. It is difficult to transform the data into an exportable that will show up, and not blow up non-CAD platforms. I know enough Civil 3D to get things where I need them to be. The ability to work on files with Civil 3D routines and then augment the process with Carlson has been a game changer.

Modules

There are modules and suites which are a collection of modules. Carlson offers a standalone 3D Program I like called Precision 3D. This Civil Suite has most of what we use and is a good place to start. It has the Survey, Civil, Hydrology, and GIS Modules.  Their Takeoff Program has an AutoCAD engine built in to get more power. The Takeoff Suite has more modules and runs on IntelliCad or your copy of AutoCAD.

File Structure

Carlson has been around since the DOS days. I’ll wait until some of you look that up. Knowing your file types is critical to doing any work in Carlson. Many of the components of a project become files on their own and you need to be sure they are all in the same folder, it will make access easier later. It takes no work to do this, just do not be alarmed when you see alphabet soup in the folder.

This example has a centerline, point files and a surface. The, somewhat old reason for this is so data does not get lost on a crash. These files were sent off the main program so not to become corrupted. When you work, they are seamlessy integrated into the program so there is no user input necessary beyond choosing what you want to work on.

Commands

Carlson and CAD commands look the same. The GUI is similar so you may not be sure who wrote it. There is nothing wrong with it except some users aren’t sure what software they are using. This seamless integration is the reason it works so well. In the video for this offering, I will show you the integration at work.

The command ribbons are separate and Carlson command ribbons may contain CAD commands that enhance the performance of the routine listed in the icon string. The icons in the top are CAD draw commands and below are the commands for Draw in Carlson.

A quick overview shows CAD is focused on different line styles and text. Carlson starts with lines but continues with symbols, sequential numbers various leaders and standards. A right-click on either companies’ menu will bring up selection options. Access is also easily typed in the command line.

Working with CAD

When we get a Civil 3D file, there is always a lot of information that needs to go away to be able to work with as small a file as possible. Bandwidth is not an issue, but we want to strip out what we do not need.

Even the smallest files can pose huge problems. A quick way to go about finding things that need attention is to do a Quick Select and see what native items are present.

Here is a small site that was loaded with objects that will not work well or at all in other software and definitely not the field hardware. We will need to convert, delete, or redraw these objects to make them work the way we want.

As an example, the existing ground for the project was a Civil 3D TIN surface. We needed that in order to do a dirt takeoff after the model was built for the contractor to confirm their numbers. A TIN Surface needs to be changed so the triangles are visable. From that point, you will need to explode the surface triangles which then become a block. Another explode turns them into 3D faces. Easy in execution but the process is not easy if you don’t know Civil 3D well enough.

Carlson has commands for converting Civil 3D entities, I have found it works with mixed results but it is always a good idea to start there. When we get as much as possible converted, we will use Drawing Cleanup to help. Several rounds of this and we are usually good to go.

Job Setup for Data

When making a set of plans, orientation is not critical. Carlson lets us set a project up in a manner that helps with data production. Getting the site georeferenced is not difficult and you can also bring in any number of online maps to facilitate data prep.

We often get footing information in structural drawings. Carlson has a Scale Wizard that will convert US to International, or back, with a click as well as being able to select a distance on the screen to tell the program the length and scale accordingly. This is good for architectural units to decimal feet.

Production

The good thing to remember is CAD is a line driven program. Drawing and manipulation lines is their strong suit. Carlson came along and added survey-centric 3D commands to make CAD a model building and production tool. I find that by using these two is a real advantage. In short, I like CAD for 2D line generation, plotting, and other graphic functions. Carlson has robust 3D commands and I learned to use them before Civil 3D, so I stay there. Others that have learned on Civil 3D like the functionality so that is their platform.

There are several ways to go about making a surface in any software. The developers have created a path to follow so we do not forget anything, but they also allow us to grab some lines and layers and make it. 3D Carlson calls their path SiteNet, it has some features that help newbies as well as us experienced builders make things look good quicker.

The drop down menu is shown here and it has what you need to make surfaces. Check their accuracy and run reports.

The ability to define target layers lets you add and remove layers for a surface. You can also use Set Layer Target that will let you click on a layer and move it to the right spot.

Many of our clients want a report formatted a specific way, the reporting options let you set things so they match the need. This is also true for the screen graphics and colors. The Display Options Command has tabs to set all those parameters in one place to make things easier. There is also an option to save and load settings, we use this for different clients to save time getting layouts correct. It is also easy to share the settings with another engineer.

 

Layers

I have always liked AutoCAD for layers. I can turn them off and on with ease and Carlson has made things even better. I will show this in the video, but I can click on a layer and isolate it. I can return to the previous state easily. Layer groups are an AutoCAD command and the Layer Manager is easy to use and edit.

In this instance, I am editing the Proposed Layer State with the actual Layer Manager Dialog box I am to use. I can make a layer visible or off, frozen or thawed, and lock if I wish. When I have done what I need, just update and the new configuration is saved.

There are also layer lists that get generated in Site Net, but not shown in the CAD command, the two can be used independently. I find I use the CAD layer states for initial work on getting rid of things that I know I will never use, then start placing layers on their targets to start the build.

There are a lot of things that are important when building data regarding layers. I want to see what I need to work with and a look at a layer to keep it or dump it. Color and line type changes are no big deal in CAD. Their DNA is around lines and it is powerful.

3D Aspects

The jump from plan preparation to a model is a big one. Other programs are model based and struggle to plot a set of plans. The gap Carlson filled was to get a great program into three dimensions. The surface tools do a good job of letting one review a surface and prepare it for the field. The elements that make up a surface are easy to prepare or convert. Working on a surface is possible in plan view by Using Viewpoint Settings to tilt and twist the plan view screen to a perspective that helps editing.

Here is a surface that has been tilted so I can make sure the contours are correct in these ponds. Normal editing takes place and gives the user a better look at what the small changes to a curb or swale can make to a model.

Some Thoughts

Data Builders come in two types, those who know Autodesk products and those who do not. Carlson is a no brainer for CAD users, things are easy to learn and you just need to add to your skill set. Non-CAD users will need to learn a lot of commands, but resources are available, especially for AutoCAD education. There is no need to separate your learning. CAD and Carlson are seamless so just jump in and do some work.

Trimble Business Center: Tips and Tricks

Trimble Business Center: Tips and Tricks

When you are learning software, there are two approaches; menu driven and process driven. Software developers write commands so that understanding software revolves around the command(s) that fit the need. My approach is to find the shortest and straightest line to the finish. Processing data, for instance, can take 15 to 30 commands. I want the most direct path to the result. This often varies widely depending on factory or dealer training.

I will highlight some of the commands that I use for data prep that can make things go smoother. This is a good way to verify if you can streamline your process.

CAD Elements

The first thing I like to do is get what I want on the correct layer group. This also brings up the discussion of how to build a surface. Do you take some layers and create a surface, or do you go through categorizing layers to let TBC create the surface? I have had many discussions, regarding this subject. Taking layers to create a surface doesn’t take a lot of additional time, and when the client asks for a takeoff, it becomes a no brainer. My world is mostly solving problems and not production. I will create a quick surface to try an idea or solve an issue. My thought is to play it by the numbers and check off the layers you want to be included in the surface.

To make this easier, I will name the layers what they are and dispense with the engineer’s goofy wording. Just go to the layer manager and type in a new name.

These layer names all now have been changed to CAD Standards. Having the same name used in every file is something everyone should do.

Where are we

At this point I have renamed the layers that have what I want for my model and linework. From there I have categorized them according to the places they need to be. Be assured when I get to this point, the list is dynamic. I will move things in and out of the groups to make things work better. This is where some of you may be thinking this process is not as good as including a few selected layers into a surface. I have done time and motion studies and it becomes a coin toss, but as I stated earlier, it sets you up well for a takeoff and changes to the site as the job progresses.

3D Elements

A surface is composed of three basic things: 2D lines, 3D lines and points. They are linked together to form a TIN, Triangulated Irregular Network. Choosing what goes into the mix is the rub. I want detail but not so much to clog the screen and drive me crazy when the time comes to fix things. In the old days, we used to worry about large surface files. They would slow down machines and make things jumpy. Now the control boxes create page files that are no longer an issue. Data collectors can still be a victim here, so it is wise to be aware of the surface file size.

Linestrings

Love or hate the name, we live and die by linestrings in TBC. When creating lines for a surface, I want to make sure all the lines I am dealing with are linestrings. Luckily, there is a TML (Transport Mapping Layer) for that, and you need to use it for the layers that are going to be a part of a surface.

Use the details that can be added to a linestring to make it stand out. Go beyond the layer name. You can see here I have named the lines that will be my islands. They reside on the edge of pavement layer, but I can select them by name if I want to do something with them later. Line style is a bit of a problem. TBC is not good with handling a lot of different line styles, so we default to “show all lines as solid” in the icons at the bottom of the screen.

I will sometimes delete the layers that I do not need or change their line type to solid to make graphics happy. One of the major threads you see in this is the adaptation of the files and work you do to create the outcome you need. Therein lies the difference in training styles – I need to get this out the door, not explore the merits of the new useless command.

Linestring Tips

Here are some tips that make things easier with linestrings.

  • Right click is your friend. Go into the coordinate box of edit a linestring and right click. A lot of options for COGO (Coordinate Geometry) come up.
  • You can do math inside an instruction. In this example I did a bearing/distance segment. The bearing is entered but I need to add two distances. I am lazy so I just do the addition in the command line:
  • Arcs are easy to do with a lot of options including smooth curves and best fit arcs. There are also detailed COGO routines for curve functions that require exact information. The quick arc to clean up an area is a big help.
  • The Browse feature lets you look at the history of a line and drill down to fix issues.
  • Linestrings have two separate and editable attributes, horizontal and vertical. You can change the way a line looks on the screen and keep the vertical elevation points you held when entering them from the plans.
  • Elevations do not need to be numbers. You can use deltas or percent slope.
  • The toggle segments button in edit linestring will make everything smooth curves.

Point Density for Surfaces

When I build a surface, I like to enter the least amount of information possible. Here is a typical scenario:

  • Enter elevations of contours to get things started.
  • Elevate curb points to match the plans.
  • Add points that look like they are necessary.

At this stage of the game, we have the minimal information to build a surface. Note that the arcs have become segmented because an arc cannot be elevated. This creates a mess, but I want you to see the process of building a surface.

 

Here we see the straight line that crosses the arc for the curb line. I have adjusted the settings to not create breaklines in the arc, but I see the elevations and can verify I am not out of tolerance anywhere.

When I am satisfied with the basics, I will let the program do two things.

  1. Add breaklines to the arcs and clean up the curves.
  2. Insert points along straight segments at the interval I want. For small sites I will go out to 10 feet. As the area gets bigger, I will set it to 20 feet as to not choke the controller.

These changes can be made on the fly and adjusted for results. You will find them in Settings/Surface/Computations.

These are the settings I will use for this project. A half a tenth on horizontal and vertical node placement and no more than 10 feet between points along a straight line.

Here is that previous train wreck fixed with updated settings.

I do not always swing point placement settings this far out when building a model. There are times when you want to see the information you inserted and see where the problems are. The fact is that any interpolation of the elevations is a direct result of what you did in the first place so the errors can be fixed because you did them. Go back to the part that does not look good and get it correct.

The big thing to remember here is the linestring that needs fixing only has the elevation points you entered so the repair is easier.

If you look at the elevations called out on this image there are just two typed elevations with interpolated points between. You can also approach a surface this way. When there is a lot going on, this can get quite busy with elevations and make it hard to see and fix.

Surface Tools

I love when I first get a surface made. Often, things are looking good but there is always room for improvement. Here are a couple things I do to check quality.

Contours

I can use contours to get a rough idea of how things look. To really get detailed, I will make the interval a tenth of a foot. No need for multiple colors, labels, or layers. You can get a good idea where to focus your energy for repairs. The image above shows a clean movement of water in this intersection. It took some work to get there and it paid off. The paving will look nice and it will move to drain correctly. Always go back to one-foot intervals after you are done for a basic sanity check. Don’t be alarmed if your contours do not match the plans exactly. Any tweaking you have done will alter the location of the one-foot lines.

Breaklines

A breakline is nothing special. It is just another name for a 3D line. We call them that because during the days of point surveys we forced a break along a line in the surface, usually at the top and toe of slopes. The name stuck and now they are used to improve a surface’s performance.

I like to use the plan and 3D views to get instant feedback. Make sure the surface rebuild method is set to auto. You will get a chance to try different ideas. As you do more of this there will be clear winning ideas for breakline placement. Pay attention to your process to reduce lines that do not help.

 

 

X