133 lines
7.3 KiB
HTML
133 lines
7.3 KiB
HTML
<!DOCTYPE html>
|
|
<html lang="en"><head>
|
|
<title>GoatPr0n.farm</title>
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<meta name="format-detection" content="telephone=no" />
|
|
<meta name="theme-color" content="#000084" />
|
|
<link rel="icon" href="https://goatpr0n.farm//favicon.ico">
|
|
<link rel="canonical" href="https://goatpr0n.farm/">
|
|
|
|
<link rel="stylesheet" href="/css/bootstrap.css">
|
|
<link rel="stylesheet" href="/css/bootstrap-responsive.css">
|
|
<link rel="stylesheet" href="/css/style.css">
|
|
|
|
</head><body>
|
|
<nav class="navbar navbar-inverse navbar-fixed-top">
|
|
<div class="navbar-inner">
|
|
<div class="container">
|
|
<button type="button" class="btn btn-navbar" data-toggle="collapse" data-target=".nav-collapse"></button>
|
|
<a class="brand" href="https://goatpr0n.farm/">GoatPr0n.farm</a>
|
|
<div class="nav-collapse collapse">
|
|
<ul class="nav">
|
|
|
|
|
|
|
|
<li>
|
|
<a href="/posts/">
|
|
|
|
<span>All posts</span>
|
|
</a>
|
|
</li>
|
|
|
|
|
|
</ul>
|
|
</div>
|
|
</div>
|
|
</div>
|
|
</nav><div id="content" class="container">
|
|
|
|
<div class="row-fluid navmargin">
|
|
<div class="page-header">
|
|
<h1>Can I haz ur dataz - Fri, Feb 1, 2019</h1>
|
|
</div>
|
|
<p class="lead"></p>
|
|
<h2 id="remote-data-acquisitions-over-ssh">Remote data acquisitions over ssh</h2>
|
|
<p>To get your data trough <code>ssh</code> to your local storage, you simply use pipes. It does not matter if you use <code>cat</code>, <code>dd</code> or any other command line tool which outputs the data on standard output (<em>stdout</em>).</p>
|
|
<h3 id="using-cat">Using <code>cat</code></h3>
|
|
<p>When using cat the there is no need to add any additional parameters in your command chain. A simple <code>cat <input></code> will suffice.</p>
|
|
<p><code>Cat</code> does not have any progress information during read operations.</p>
|
|
<h3 id="using-dd">Using <code>dd</code></h3>
|
|
<p>The program <code>dd</code> requires more user interaction during the setup phase. To use <code>dd</code> you have to give the <em>if=</em> argument. The use of different blocksizes (<em>bs</em>) will not have that much of an impact on the speed. Feel free to have a look at this <a href="https://goatpr0n.farm/2019/01/23/what-if-the-cult-of-dd-is-right/">this</a>.</p>
|
|
<p><img src="https://dl.goatpr0n.events/$/VAN9L" alt=""></p>
|
|
<p><em>Wow. So scientific! Much CLI.</em></p>
|
|
<h2 id="examples">Examples</h2>
|
|
<p>A simple example with no output to the terminal except of errors. The output to <em>stderr</em> is not suppressed.</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#75715e"># Using cat to copy /dev/sda</span>
|
|
$ ssh <user@remotehost> <span style="color:#e6db74">'cat /dev/sda'</span> > sda.raw
|
|
</code></pre></div><p>If you want to suppress errors, use:</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#75715e"># Using cat to copy /dev/sda</span>
|
|
$ ssh <user@remotehost> <span style="color:#e6db74">'cat /dev/sda 2>/dev/null'</span> > sda.raw
|
|
</code></pre></div><p>The next example will demonstrate the use of <code>dd</code>.</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#75715e"># Using dd to copy /dev/sda</span>
|
|
$ ssh <user@remotehost> <span style="color:#e6db74">'dd if=/dev/sda'</span> > sda.raw
|
|
</code></pre></div><p>Of course you can suppress errors as well.</p>
|
|
<h2 id="speeding-up-the-data-acquisition-and-minor-tweaks">Speeding up the data acquisition and minor tweaks</h2>
|
|
<p>With the basics covered, we can begin optimizing our data transfer. In the first step we speed up the transfer with <code>gzip</code>.</p>
|
|
<p>The argument <em>-c</em> will write the compressed data to <em>stdout</em> which will be piped to your local system.</p>
|
|
<p>Of course you can use this with <code>cat</code> as well.</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ ssh <user@remotehost> <span style="color:#e6db74">'dd if=/dev/sda | gzip -c'</span> | gunzip > sda.raw
|
|
</code></pre></div><p>To add some progress information, you have two options with <code>dd</code>.</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#75715e"># dd status</span>
|
|
$ ssh <user@remotehost> <span style="color:#e6db74">'dd if=/dev/sda status=progress | gzip -c'</span> <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span>> | gunzip > sda.raw
|
|
</code></pre></div><p>The <em>status</em> argument writes the output to <em>stderr</em> and will not end up in your local copy.</p>
|
|
<div class="highlight"><pre style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-bash" data-lang="bash">$ <span style="color:#75715e"># dd through pv</span>
|
|
$ ssh <user@remotehost> <span style="color:#e6db74">'dd if=/dev/sda | gzip -c'</span> <span style="color:#ae81ff">\
|
|
</span><span style="color:#ae81ff"></span>> | pv | gunzip > sda.raw
|
|
</code></pre></div><p><code>Pv</code> needs to be installed separatly. Check your packet manager. 0r teh Googlez!</p>
|
|
<p>KTHXBYE.</p>
|
|
<h4 id="update-01">Update #01</h4>
|
|
<p>Fixed a problem within the examples. Had a pipe too much. #Fail</p>
|
|
|
|
<h4><a href="https://goatpr0n.farm/">Back to Home</a></h4>
|
|
</div>
|
|
|
|
|
|
</div><footer class="container">
|
|
<hr class="soften">
|
|
<p>
|
|
©
|
|
|
|
Julian Knauer
|
|
|
|
<span id="thisyear">2020</span>
|
|
|
|
</p>
|
|
<p class="text-center">
|
|
|
|
|
|
|
|
|
|
|
|
</p>
|
|
</footer>
|
|
|
|
<script src="/js/jquery.js"></script>
|
|
<script src="/js/bootstrap-386.js"></script>
|
|
<script src="/js/bootstrap-transition.js"></script>
|
|
<script src="/js/bootstrap-alert.js"></script>
|
|
<script src="/js/bootstrap-modal.js"></script>
|
|
<script src="/js/bootstrap-dropdown.js"></script>
|
|
<script src="/js/bootstrap-scrollspy.js"></script>
|
|
<script src="/js/bootstrap-tab.js"></script>
|
|
<script src="/js/bootstrap-tooltip.js"></script>
|
|
<script src="/js/bootstrap-popover.js"></script>
|
|
<script src="/js/bootstrap-button.js"></script>
|
|
<script src="/js/bootstrap-collapse.js"></script>
|
|
<script src="/js/bootstrap-carousel.js"></script>
|
|
<script src="/js/bootstrap-typeahead.js"></script>
|
|
<script src="/js/bootstrap-affix.js"></script>
|
|
<script>
|
|
_386 = {
|
|
fastLoad: false ,
|
|
onePass: false ,
|
|
speedFactor: 1
|
|
};
|
|
|
|
|
|
function ThisYear() {
|
|
document.getElementById('thisyear').innerHTML = new Date().getFullYear();
|
|
};
|
|
</script></body>
|
|
</html>
|